I'm still learning....

If you have Jetpack (& other plugins) installed but not activated, will that slow down your site?

If you have images uploaded in Media Files but not used on the site, will that slow down your site?

If you have several posts in draft mode, but not published, will that slow down your site?