I've never heard that Jetpack was slow. They have a great team of developers that will try to put out the most efficient code possible. I'm not sure why that's even in question. What I've seen from most people talking about Jetpack's bloat has been in the context of the plugin having too many features, not its speed.