I know this would irritate the heck out of me on my dev install, which currently has 146 themes installed. On my personal site, not so much. Then again, I don't actually use the customizer on my personal site.