In reply to Li-An.

I also love markdown.

Personally, the tools I use 95% of the time in Jetpack are:
Markdown
Stats
Publicize
Photon

It's super lightweight when you only have a few modules activated. We can't get into testing every combination, but I would be surprised if running just Markdown caused any sort of significant jump in load times over just plain WP core.