By default, WordPress doesn’t notify users when their comments are approved from the moderation queue. If you’d like to change that, consider using the Comment Approved plugin by Niels van Renselaar. The plugin is simple to use and configure. After it’s installed and activated, you’ll find the settings in Settings > Comment Approved. It’s important to note that even when the plugin is activated, it won’t send out notifications unless you check mark the box to enable the comment approved message.

The text area is populated with a default message using the only two shortcodes available. You can customize this message using standard HTML. Here is what the default message looks like in an email.

While I like the default behavior to notify users when their comment is approved, you can use this plugin to award first time commenters. For example, you can send them a link to a free eBook or a different promotional item. Once the user’s comment is approved, they won’t see the approval message again.
It’s strange that comment approval notifications are not part of the default behaviour of WordPress. I think it makes sense, especially if the front-end notifies them that their comment may be held in moderation. While I doubt this plugin will prevent commenters from getting in touch with site administrators to figure out why their comment is not displaying on the site, at least they’ll know when it’s approved.
Comment approvals could rapidly become annoying if they were there enabled by default. I comment on a lot of blogs and would have to start blocking those notifications if they arrived every time. It’s already a hassle having to respond to comment subscription emails all the time.