Cancel

Sarah Gooding

ThemeCheck.org: A Free WordPress Theme Validation Service

ThemeCheck.org is a new site that provides a free validation service for WordPress and Joomla themes. Visitors can upload themes and templates and the service will analyze the files and run tests to verify security and code quality. The site gives each theme a quality score along with a list of alerts and warnings.

For example, the Responsive theme, which has been popular on WordPress.org, gets a validation of 95%. Automattic’s Circa theme validates 100%. Critical alerts can bring a theme down to a zero validation score.

The site displays scores for recently checked themes, both free and commercial:

themecheck

ThemeCheck.org creates a badge for each theme to display its validation score and link back to the report. Theme authors or sites that distribute themes and templates can embed the badges on their websites.

The validation service uses a forked version of the Theme Check plugin to validate themes. This plugin allows WordPress theme developers to make sure their themes are up to the latest theme review standards. It’s the same tool used for automated testing of new themes submitted to the WordPress Themes Directory.

After finding that the service was not able to accurately score WordPress child themes, I spoke with Guillaume Baudhuin, the site’s creator. He responded to my concerns by updating the validation service in the following ways:

  • Changed the score calculation algorithm. It is more severe with critical alerts counting for -20% and Warnings counting for -1%.
  • Some critical alerts were downgraded to warnings (in checks for screenshots, metadata, etc.)
  • Child themes are managed correctly. Some checks no longer apply to child themes.
  • Improved management of template overrides

It’s not surprising that the service would have problems with child themes, since it uses a fork of the Theme Check plugin. Unfortunately this plugin doesn’t accurately account for child themes and will throw a bunch of warnings that often do not apply. I’d be curious to know if ThemeCheck.org’s changes regarding child themes could be applied to the original Theme Check plugin.

A Useful Tool for Consumers Shopping for WordPress Themes

validation-scoreThe team at ThemeCheck.org seems dedicated to providing the most accurate validations possible, given their attentive response to concerns raised by their users. Scrolling through the recent results you can see that most of the themes are free, but there are a few commercial themes peppered throughout.

While anyone can validate a WordPress theme using the Theme Check plugin, many don’t want the hassle. This service makes it convenient for anyone to perform a validation. If you use it to validate a theme and you don’t want it to store any information about the theme’s score, you can select that option when uploading.

ThemeCheck.org could potentially be a very useful site for consumers who are shopping for WordPress themes, if more commercial themes are added. It empowers non-technical consumers to approach theme developers about the critical warnings discovered through the validation tool. In the future, the team plans to add an option to sort themes by score, i.e. to display all themes that score 100%.

If you have any feedback about the service or want to inspect the code it uses to perform validations, ThemeCheck.org is open source and hosted on github.

Category:
Tags:
27

27 responses to “ThemeCheck.org: A Free WordPress Theme Validation Service”

  2. Hmm… I just tried to run Oenology through the service, but someone had already done so. It only got a 99. :/

    It seems that the site has escalated the severity of an INFO check (use of include()/require()) to the level of a warning. The end result is that a legitimate (and proper) use of include()/require() – to include functional files in functions.php – causes the rating to be downgraded.

    So, there’s some room for improvement there. But, in all fairness, there’s room for improvement in that check in the Theme Check Plugin as well. That INFO output should only be given if include() or require() are used in template files. The use of include() and require() should be ignored when used in functional files. So, that’s not entirely the fault of the themecheck.org site (though I’m unsure why an INFO check is being escalated in severity).

    Create forum topic from comment

  3. Great article ! Chip, tolerance level on includes/requires is a tough question because it’s hard to make everyone happy. Differenciating between functional and template files seems to be an interesting solution. I just have to find a secure way to differenciate those two kinds of files.

    Create forum topic from comment

  7. The general idea with this ThemeCheck service is cool & useful. I like it, and despite some reservations, will continue investigating.

    Certain of the implementation-details that have been selected, look like they are or invite ‘issues’. Certain of these are readily addressed/fixed, while others; maybe not-so-easy.

    Calling what is done here “validation”, and then asserting that “security and code-quality” objectives have been achieved, is overstating what it is & does. This terminology & claim poses the service as something more-authoritative than it is. “Validation” of code is a term that already has an established meaning; it’s an advanced & challenging (‘impressive’) field of computer science. It’s virtually impossible than any ‘real’ validation is being done.

    So what is the ThemeCheck website doing, then, if not “validation”? Well, this service is of course based on a top-ranked WordPress plugin called Theme-Check, written by Simon Prosser and Samuel Wood (Otto). On their Theme-Check FAQ page, they describe in more detail what it does:

    … The theme check is designed to be a non-perfect way to test for compliance with the Theme Review guidelines. Not all themes must adhere to these guidelines. The purpose of the checking tool is to ensure that themes uploaded to the central WordPress.org theme repository meet the latest standards of WordPress themes…
    …
    This theme checker is not perfect, and never will be. It is only a tool to help theme authors, or anybody else who wants to make their theme more capable. [emph. added]

    That’s a very humble description, compared to ThemeCheck’s terminology & claims. Toning down their highfalutin language shouldn’t be too hard. (I would check with Mr. Prosser & Mr. Wood, and ask to quote them & their FAQ-language. Those are heavy-hitters at WP, and their ‘implied endorsement’ is meaningful & valuable. )

    Next: the numerical scoring system. This is where the overall working-design of the ThemeCheck website might be harder to tweak.

    When we say things about products in the market, or more pointedly, when we publish statements about things & people, we fall under moral & legal obligations. We can’t just up & trash-talk the products of other entities, without responsibilities & consequences.

    If we are going say negative things about products, we have to have our ducks in order. It has to be real & verifiable, if we are going to eg rate the Ford Super Duty at 0, and the Dodge Cummins Ram at 100. Ford could take a dim view of that, and they do have recourse.

    To the extent that this scoring system is arbitrary & subjective – that numeric valuations have been ‘pulled outa thin air’ and assigned to points of Guidelines deviation & compliance – the ThemeCheck website is on shaky ground. They’re ‘makin it up’ – and can be challenged, successfully.

    Somebody is likely to ‘take exception’ to how their theme is be posed, at ThemeCheck. They have the right to ask that an accounting be forthcoming. People who publish claims & characterization about products are, after all, held to be morally & legally accountable.

    It would be a big surprise, if this scoring system is a consistently accurate or meaningful reflection of the usability & merits of themes. Best to tackle that weakness, sooner rather later.

    Other than than … nice idea, and best of luck with it!

    Create forum topic from comment

    • I only wish this statement where true, “Somebody is likely to ‘take exception’”

      Sadly if you ever tried to get someone in another country to retract a statement; or even get a web host or other service provider to take down a site for overt copyright infringement, you may have experienced how little this “someone might do something if you do” phrase has to do with reality–about as much teeth as my sons 1 year old Bearded Dragon lizard…

      Create forum topic from comment

      • True: the ethical & legal often get short shrift on the net.

        But, the net can also do an impressive ‘judge, jury & executioner’.

        Especially when trying to get established with a strong self-identifying (competitive) net community – like WordPress.

        Once members with a meaningful voice decide they don’t like an activity or service – they don’t talk to lawyers or write essays on ethics – they just come around broadside and start pounding away.

        I don’t think that is necessary or called-for, certainly not at this stage. The Theme-Check site is a good idea, basically, and it could be a upgrade for WordPress generally, and for themes in particular. Themes need more attention; more acclaim … plugins are kinda hoggin’ the glory these days.

        I figure a shot or two across the bow is plenty-good for now, just to clarify proprieties.

        Create forum topic from comment

    • I’ve been thinking a lot about your point : “Calling what is done here “validation”, and then asserting that “security and code-quality” objectives have been achieved, is overstating what it is & does. This terminology & claim poses the service as something more-authoritative than it is.”

      I mostly agree with your point of view. Some claims definitly need to be adapted. This service is meant to be trustable, and as such, it should not over-promise what it does. However, the main goal of themecheck.org is to offer an easy and quick service. So, the claims also need to be easily understandable… We’re working on it.

      As it’s been said here, the service is new and we need time to adjust it, in a better way for everyone.

      People’s remarks on this page are of great help.

      Create forum topic from comment

      • In most uses of ‘actual’ validation, it is paired with a verification phase. It is a dual process, usually seen as “verification & validation”.

        Let’s consider eg the verification & validation of a pudding recipe (which is a program for converting ingredients into a define outcome or product, under specified manipulations).

        Eggs: check. Milk: check. Flour: check. We have now verified the correct ingredients.

        Oven preheated 350 degrees F: check. Ingredients combined in the given order: check. We have now verified the proper preconditions for program execution.

        To perform the validation phase, cook the verified ingredients: execute the program. Did the program-execution result in pudding? ‘The proof is in the pudding’. Do not use children to judge (or test) the pudding: use your in-laws, preferable your mother-in-law. If it’s pudding, the recipe is validated.

        To validate code, it must be executed. You can’t know if it produces the intended pudding, unless you run it.

        Verification is static. In an airplane version of pudding, we verify that the plane is ‘ready’ or ‘prepared’ to fly using a preflight checklist (while parked static on the tarmac). The case of an aircraft clarifies that the proof (or validation) is dynamic: it is in the flying. The preflight is the verification, in an airworthiness V&V.

        The WordPress Themes Guidelines appear to be a preflight checklist. The ThemeCheck plugin appears to be an automated tool, for conducting the static preflight or (pre-execution) verification component of a two-phase theme verification & validation.

        And then, verification & validation itself is but one phase in various multiphase processes that aim to establish eg code quality, and security, among other states & metrics.

        Create forum topic from comment

  9. I wouldn’t be too hard on the service. It’s new. Give the guy time to work out the issues.

    I neither endorse the service nor disparage it, because honestly I’m not certain whether or not it’s a good thing. I’ve looked at the code he has on github a bit. There’s potential there.

    On the whole, if it helps people make better themes, then that’s good. But the site seems more geared towards providing a numeric service for users to use, and I’m not sure that that is necessarily the best approach to take. A user shouldn’t be choosing based on code quality, but on design quality. The code quality should be a given, for all themes. We’re not there yet, but still.

    Back when we first started doing theme checking via automated means, the first tools pross wrote were like this: upload the code, have a process scan it. I rejected that at the time in favor of a plugin because it lets people check their own code, on their own test sites, and not having to rely on a central service. I still think that is more useful for developers, to be able to give them the tools directly instead of having the tools yourself and requiring them to jump through hoops for information. So what I’ll probably do is see where he goes with it in the code, and then steal all the best ideas for incorporation back into the Theme Check plugin. Makes the most sense. :)

    Create forum topic from comment

  10. Very cool idea, I like the concept and theory quite a bit.

    However, after testing a couple themes and looking at the test results of other themes it seems that some updates to the code need to be addressed.

    My themes first run through resulted in a very bad score, I host its code on github and therefore it had the .git folder included. While it’s good to warn that a hidden file is included with the theme, I am thinking that it would be better if it checked for malicious code within those files.

    Also, some of the things that knock down the score aren’t related to whether the code is good or the security of the theme, rather these warnings are related to theme author preferences.

    I can’t wait to see the updates and improvements. This is a very cool idea.

    Create forum topic from comment

  11. This can be a useful service if he’s providing checks that aren’t present in the plugin, but I think it needs some changes before it can be useful as a user-facing service. If the target audience is theme purchasers, the vast majority won’t be able to judge the merit of any particular error/warning, so they’ll need a lot more hand-holding on what really matters.

    Theme Friendly’s format has more potential as a user-facing service. It provides separate scores for errors and warnings, provides a manual review score, and lists features separately so that missing features aren’t interpret as “problems”. It also has a section where issues can be explained. This requires more work, though, so it’s not as simple as an automated scanning.

    Create forum topic from comment

  12. Really useful service this. Kudos to Guillaume for taking the useful Theme Check plugin and making it more widely available. I did notice a few themes that I tried, it said it they didn’t have a theme name in the css file, even when they did. No doubt these little issues will get sorted out over time though.

    Would also be good if it allowed for different versions of a theme. Currently if you try to validate a theme and another version has already been validated, it wont save the results.

    Thanks for sharing this great service Sarah and well done Guillaume!

    Create forum topic from comment

You may also like

Newsletter

Subscribe Via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.