Why not start small? Some script/program which scans for typical mistakes. Initially it could just grep for things which are a no-go but are known to be used, maybe also some typical novice-mistakes, and some bad-programming-style things. Maybe also grepping for things which should be there but are forgotten from time to time.

With this as a base (and I'm sure as time increases the things which are looked for in such a program will increase), the basic quality could improve for the worst plugins. Provide this to the people to check before submission of the plugin, so that people can check locally, and also run them for each new submission to the plugin-repository. Initially you could just issue a response from the output of the program to the authors, later you could maybe block based upon the output.

As an example what is possible with such a script you could have a look at "portlint" (http://www.freebsd.org/cgi/cvsweb.cgi/ports/ports-mgmt/portlint/src/portlint.pl), the script which checks the meta-data of the FreeBSD package management system.

I'm sure as time passes such a script/tool would receive improvement submissions and extend into something which catches a lot of problem cases.

I'm aware that this will not be able to find everything you want to find, and that some things may even be imposible to find with such a tool, but you can raise the bar by a good amount to what is done in the plugins ATM.