In reply to Andreas Nurbo.

You need to write JavaScript
You only need to write extra JS if you want to use postMesaage - which you don't have to use!
you need write configuration for the various controls and where it goes
Don't you have to write configuration using the settings API in order to create an admin page for example? If anything, the customizer makes that easier!
you need to register the various files that loads the control
That's only if you plan on building your own advanced controls. Which you would have to do anyway if you were using the settings API... and it would be even harder there 'cause you'd also have to write a lot of extra code to support that!
You need to write code for how it should be rendered on the frontend
And if you were using the settings API you wouldn't have to write any code to handle how your saved options would affect the frontend of a site? The code to control how your options affect the theme/frontend shouldn't change at all, it's exactly the same thing.
Your solution to my complaint is that I need to depend on yet another framework?
No, it's not!! What I said earlier is that if you want to do it it can be done, and then posted a link as an example. Building a simple function to simplify the way you write your own code shouldn't be that hard, and if you need any assistance I'd be more than happy to help out!