I believe we need a toggle inside Preferences so that we can choose to hide the Patterns tab seen inside the Inserter screen. I went ahead and made this issue: Preferences: Adding a toggle to hide Patterns. https://github.com/WordPress/gutenberg/issues/36470