I've personally not had an issue with Gravatar. I think it's a good addition, but I agree, a better interface/link is needed. Rather than JetPack, it needs to be built into the core.

This looks to be a useful plugin to disable the user gravatar.
https://wordpress.org/plugins/disable-user-gravatar/