I think the most important point in all of this is if you need to "compromise" design after the fact, a11y wasn't baked into the process from the beginning. There will be issues that need to be fixed always, but you can really limit the risk by making a11y a PR approval requirement.