I really hoped that this feature was finally going to work so I could reduce my dependence on JetPack. I tried it, and I'm sorry to say it is still unusable for my sites. In my quick test, I found:

Writing your own CSS to suppress captions seems to be the only way to turn the captions off -- there should be a switch for this in the Block Options area.
It cannot cope with mixed portrait and landscape images. It has a setting so that it no longer crops images, which is very good. But instead, it just leaves large areas of white space -- which is unacceptable.

I'm struggling to understand why this cannot be made to function in the same way as JetPack Gallery shortcode, which is another freely available Automattic product that runs rings around the Gallery block.