Generally I do agree, but in reality I've seen that we needed to add specific blocks in cases where they could easily be constructed out of other blocks.

Why? Otherwise, the Block Editor was not performant anymore until a point where you couldn't use it at all in some instances. For example having 200 blocks because you use group + heading + image + button instead of having just 50 blocks (that simply combine these inputs) can have a huge impact on the backend editor right now.

This is the main reason I'm unsure how well block patterns will work out in the end. The idea is great: don't create blocks for elements that can be constructed out of other blocks. But the editor's performance needs to keep up with this, otherwise it will be unusable, as it sadly is right now for larger pages.