Someone did not think through a use case where there are MANY menus. Primary and secondary with a reasonable number of items that vary per page/post for logged in/logged out users. This would be a real PITA if in the customizer.

If WP core really wants to put it there for simple situations, do it, but just leave access in the admin for more complex scenarios.