This is just one of the very many reasons I use Postmatic, and why WordPress needs a better default set-up.

I've found Jetpack to be useless lately when it comes to notifications, and I then forget to go back and check if there's a reply or not. Also, depending on how the blogger sets up notifications, some of the comment reply plugins are underused too.

At least with Postmatic I know everyone that subscribes to the conversation can actually continue to take part in it...