I wouldn't be too hard on the service. It's new. Give the guy time to work out the issues.

I neither endorse the service nor disparage it, because honestly I'm not certain whether or not it's a good thing. I've looked at the code he has on github a bit. There's potential there.

On the whole, if it helps people make better themes, then that's good. But the site seems more geared towards providing a numeric service for users to use, and I'm not sure that that is necessarily the best approach to take. A user shouldn't be choosing based on code quality, but on design quality. The code quality should be a given, for all themes. We're not there yet, but still.

Back when we first started doing theme checking via automated means, the first tools pross wrote were like this: upload the code, have a process scan it. I rejected that at the time in favor of a plugin because it lets people check their own code, on their own test sites, and not having to rely on a central service. I still think that is more useful for developers, to be able to give them the tools directly instead of having the tools yourself and requiring them to jump through hoops for information. So what I'll probably do is see where he goes with it in the code, and then steal all the best ideas for incorporation back into the Theme Check plugin. Makes the most sense. :)