Yeah, I would be worried about version conflicts. I'm sure that is a solvable problem, but realistically, I'm not sure that using the WordPress.com systems for this is any better of an idea than using the Use Google Libraries plugin.

The main problem with offloading library loads like this is simply one of connectivity. You're relying on your site's viewers being able to connect to that external service (Google, wp.com, etc) to load the JS files. While this is relatively true in most places, in many cases it is not, which is why the default is to package files with the core, so that you have everything needed for the site to work anywhere.

With Google, they maintain version numbers as part of the request, so you can be assured that the version conflict problem won't happen. WP.com does not currently do this that I know of, although that would be easily solvable on their end.

It's likely that WP.com has more of the libraries than Google's JS library mirrors do though, so there is that.

In the end, I'd suggest using the Google libraries plugin if you want this sort of speedup, and even then it's a minor speedup at best. Most people don't use tons of Javascript on the front-end of their sites. For those that do, reducing that would be better in the long run. Sure, loading jQuery from a CDN is faster, but it can be problematic for some users.