As Chip has described, the concept of a "format" really just started as supported taxonomy to allow theme developers to apply consistent treatment across any blog (instead of one theme using post meta, one using a category, etc.).

Jump forward to 3.6 and realizing there is more nuance. What if this is a video post and my theme wants the video on top of the content and her theme wants it below? Standardized meta on that allows for more interesting things and the ability to solve more interesting problems.

To Rob's point, the initial implementation was confusing, but we (Crowd Favorite) created the FavePersonal theme and the Post Formats UI plugin because if we standardized on meta and fields, we could do fun stuff in the loop views, single views, and feed very easily (eg: put a full width image on the single view of "Image" posts and drop the sidebar, replace the permalink in the feed with the "Link URL" a la Daring Fireball or other linkblogs, etc.).

Hopefully this leads to more interesting uses of WordPress in the near future...