There is a quick way to scan the list of installed plugins for those not in the directory that doesn't require a plugin. If the "View details" link is missing from any plugin in the plugin list that means it's not listed in the directory. It's not as obvious as an alert or red flag though.

My own plugin Gauntlet Security includes a basic assessment of all installed plugins including: development activity, rating, if it needs updating, and if it's not in the official plugin directory. But I really don't think this sort of functionality is important enough to be in core - there's too much judgement involved in calling out plugins for things which could be quite innocent.

This sounds harsh but I'd also like to float the idea that ultimate responsibility is on the site maintainer who chooses what software to install on their site. Until the plugin directory is less of a free-for-all, every plugin should be vetted with great caution.