Upcoming Changes and Steps for an Overhauled WordPress Theme Review System

On Wednesday, WordPress executive director Josepha Haden Chomphosy posted the next steps forward for themes and reviews for the official theme directory. In the post, she describes the tools and types of access the Themes Team needs. She also laid out some other goals for the system. The timeline is to have much of this in place by early 2022.

Two months ago, things were coming to a head. Project lead Matt Mullenweg saw much of what we have all been seeing. Creative contributions to the free directory were few and far between, many of the submissions merely being stripped-down “lite” themes with commercial interests.

There was some disagreement on why the directory was not producing the high-quality projects users should expect from an official source. Mullenweg cited the rules and update mechanism as problem areas.

However, others like Joost de Valk, the CPO of Yoast, said the reality of the situation is that money is now a part of the equation. Producing high-quality products, maintaining them, and supporting them is not sustainable without the financial resources in place. Because WordPress.org provides no path for developers to make money directly, upsell-motived themes are the result. Eric Karkovack expanded upon this in his piece for Speckyboy, Are High-Quality Free WordPress Themes a Thing of the Past?

Some of the Themes Team members disagreed that the rules were the problem. At the heart of the team’s handbook is the idea that themes should be GPL-compatible, secure, and not break things.

The problem is not necessarily specific guidelines but the process. Mullenweg wanted to switch to a post-commit strategy that would see themes move into the directory more quickly. The goal is to be a little more like the plugin directory and let users guide others through the star-rating review system.

However, themes and plugins are different beasts. Themes must follow some standard patterns and do some specific things to actually work. The best way to make that happen is with automated tools performing the grunt work that humans have been doing for the past decade. Many guidelines could become a line of code in a script. Each new line would lighten the burden on volunteers.

The Themes Team agreed with his assessment of the theme quality. However, some did feel like the theme system was the oft-forgotten stepchild who received all the hand-me-downs from its preferred sibling, the plugin directory. They needed resources from the community to drive any sort of change. Team members had little power outside of their gatekeeping responsibilities and were short on volunteers.

Changing Hearts and Minds

Wave in the shape of a heart.

Haden Chomphosy published notes on the meeting in February. The post detailed the ideas and what took place. However, much of it seemed vague in terms of actionable items. It was the groundwork phase.

In a private discussion with one of the Themes Team reps, they said the meeting was productive not because of action taking place but through the changing of outlooks. More of the team reps warmed to the idea of reducing the requirements and moving forward with a change. The meeting was more about winning hearts and minds, which was a necessary first step.

This changed outlook did not mean throwing caution to the wind and flipping the switch overnight. The team wanted to set some guardrails in place, particularly surrounding high-priority issues like proper licensing.

“In the meeting, we discussed the need to change the review process,” said team rep Ari Stathopoulos. “All guidelines have a reason they exist. They were all added after some things got abused. But the process followed had an unfortunate side-effect; the rules that were added to avoid abuse by a few bad apples are the same rules that hinder innovation and deter people from submitting a theme in the repository.”

He brought up the universal rules of not doing evil things, disrespecting others, or abusing the system. Citing them as the foundation of what the guidelines should be. “But then, of course, everyone has a different definition of evil, disrespect, and abuse, so something a bit more verbose may be needed but obviously not as verbose as the dozens upon dozens of guidelines we currently have.”

The Next Steps: Tools and a User-First Strategy

Workbench with various tools sitting on it.

The first goal is to have access to a functional meta environment for testing. One of the team reps currently has this. However, others would need access in the long term. Moderator tools are also on the list for reviewers, likely similar to what the Plugin Review Team has.

Those are some of the baseline things. The next item will be more automation. Dion Hulse is currently working on automated security checks, which should help with a consistent problem area. Steve Dufresne is working on an automated code-scanning solution.

One idea for a post-commit strategy is flagging themes with “quality tags.” These include items like Gutenberg-ready, security, last updated, translation-ready, and accessibility. It is not clear how this system would work, but it could be a way to surface themes in the directory that meet standards. Perhaps a new featured-theme algorithm should be in the works?

The last piece of the proposal introduces the concept of a yes/no voting mechanism for end-users. These would be “trust tags” that allowed users to mark themes as updated, visually broken, and more. The goal is to hand over much of the gatekeeping responsibility to users, putting them in the driver’s seat of what they want out of the theme directory.


19 responses to “Upcoming Changes and Steps for an Overhauled WordPress Theme Review System”

  1. Firstly, I want to say that what I’m seeing from Josepha is highly encouraging, and the first real progress toward meaningful change in years.

    Having said that, I’m glad to hear that the Theme Team pushed back when Matt said this (which is complete rubbish): “The .org theme directory rules and update mechanism have driven out creative contributions, it’s largely crowded out by upsell motived contributions.”

    Contrary to Matt’s assertion, you have nailed the real issue: you simply can’t separate the financial motives behind most/all of the developers who are producing the highest-quality Themes, and the Theme directory nor the Theme Team has ever had the infrastructure to accommodate such motives or such developers.

    As a result, the shining-star Themes and theme developers got lost in the deluge of bad code, bloatware, and copycat Themes being submitted. (And that doesn’t even get into the legitimately bad actors, who could much more easily slip through the cracks.)

    Reviewing such Themes manually takes a non-trivial amount of time. (Remember: no real, automated tools for most things. ThemeCheck helped, but it really wasn’t sophisticated enough to know if core functionality was incorporated/hooked/filtered and implemented properly, etc.)

    Most of the Theme Team wanted either to help new developers improve their Theme development skills, or to use the Theme Team to improve their own skills – laudable goals that I hope remain today. Unfortunately, all that time spent buried in the deluge left little time for helping people.

    I also disagree with the suggestion that the guidelines somehow stifle innovation. The guidelines enforce following core WordPress coding practices and implementation. Themes should not encourage “innovation” in a way that breaks core functionality, code, and methodology. Innovation should come through implementing core functionality, code, and methodology in innovative ways.

    Ultimately, a huge part of the answer to this impasse is improving the underlying Theme submission and automated vetting infrastructure, process, and code checking. (Again: something the Theme Team has been requesting for years.) Allowing direct SVN commits, with a post-commit vetting process (I repeat myself: something the Theme Team has been requesting for years) will remove barriers for experienced Theme developers, and will facilitate and encourage iterative remediation of Theme issues and continuous improvement of submitted Themes.

    So, I look forward to see what Josepha, the Meta team, and the Theme Team can implement to make these improvements a reality.

  2. I don’t expect to do a lot of theme revisions, don’t want a lot of complex features, so having basic page options (various column set ups, etc.) and typographic and spacing control are most important. I will generally set up a child theme to change colors, typefaces, etc.
    Certainly some complexity/detail work/lots of stuff I don’t know about is needed for security issues, and I rely on WordPress and themes for much of that, and plug-ins, but other than that, simple (and hence inexpensive for theme makers to create) works for me!

  3. There is not much to offer for the .com WordPress users. Seems there should a variety of themes for new users that are easy to understand, but not to the point of being boring. Not all users are techie gifted and some have trouble understanding the new editor so I can’t imagine them attempting to build their own theme. A theme that functions well, has all colors available even for backgrounds, and for some a sidebar can be simplistic, but has enough bells and whistles to spark one’s creativity. IMO, an attractive theme is an excellent lure for newbies so they can easily showcase their work.

  4. Too many changes and updates required due to continuous rollout of unfinished fast moving target Gutenberg in core leads to abandoning of free themes by authors.

    This outcome was crystal clear from the beginning for everybody who saw the rollout concept, many have warned that exactly this would happen.

    Free community work of authors has been overstressed since WP 5.x, authors now move on to lite/paid to cover the required extra time for Gutenberg or simply abandon everything. Thats same for plugins.

    As simple as that.

    • The issues this article refers to pre-date Gutenberg by years. I know. I was on the review team, and this is not a new discussion. You can certainly argue that the block system has exacerbated the problem to some extent for some authors (it’s not even a requirement to support it), but it being the root cause is simply not the reality of the situation.

  5. The solution is easy. Don’t allow free themes that have an up sell to be listed as free. People can search the paid area for an upgrade is need be.

    Or the obvious solution is for WordPress to actually make a good default theme for once. A theme people are not ashamed to use.

  6. This is definitely encouraging news. Let’s make it easier for the Theme Review Team to do their thing, while developers can do theirs.

    It seems like the theme directory has languished for so long. Hopefully it improves to the point where the really good free themes are easier to find, thus giving developers more incentive.

      • If you’re in it strictly for the money, that’s fine – you don’t have to contribute a thing to a free theme repository. If you want to give something back to the community and maybe get your name out there, then you can participate.

        There are a lot of developers out there and each has their own opinion. But the bottom line is that these improvements to the directory are welcomed.

  7. I am a WordPress hobbyist in the true sense of the word. I have released one theme into the .org repository (I stopped work on it after it was removed from the directory in mid-2016 for not using the Customizer) and a handful of plugins (of which I have been actively maintain one since 2011). All my projects are and have been 100% free, both in terms of code and cost; I have a “coffee” button tucked away but that is only for people who insist on paying for my code.

    I use the term “hobbyist” because WP has nothing to do with my day job and I code with it purely to stay abreast of web technologies and terminology. When it comes to putting food on the table, WordPress is akin to a rounding error in my case. My guess is that there are a few others like me, though I doubt that the number of such people would go beyond double-digits.

    As a hobbyist, in the past I found the theme directory guidelines getting exceedingly draconian with every passing month. To me, the crux of the problem lay in WP itself not building things into the core for a long time and preferring the mantra, “There is a plugin for that”.

    As an example, Menus were introduced in WP 3.0 in June 2010 (adapted from Woo’s offering), but for a long time there was no way to automatically add pages or categories to it. In the meanwhile, since menus are considered design elements, most themes had to come up with ways to manage menus. As a net result, when the native menu system finally matured a few years later, theme developers were asked to remove their own implementations in favour of the native. The native feature was a welcome one, but it would have been so much better if the initial release in 3.0 was more full-featured.

    Similarly, the customizer was introduced 2 years later in June 2012, by which time pretty much every theme had its own options panel. Some options panels were so comprehensive that moving to the customizer felt like a massive regression as far as the interface was concerned, never mind the talk about live updates being visible. A paraphrased conversation between the developer and the review team typically went this way:

    Dev: My theme’s options panel is very complex. Recoding it to fit into the customizer will take a lot of effort.
    WPTRT: It can be done. You have to do it.
    Dev: What takes 2 clicks to reach now will take 6 clicks in the customizer. Going from one option in one panel to another option in a different panel requires a user to navigate all the way back to the top, then navigate down to the other option.
    WPTRT: Think of a user coming from a different theme – they will be used to the customizer.
    Dev: The live update feature will take a long time to code for the number of options that I have.
    WPTRT: You don’t have to code for live updates.
    Dev: Then what is the use of ripping apart my whole framework?
    WPTRT: Again, think of a user coming from a different theme – they will be used to the customizer.

    The desire to use the customizer might have been a noble one, but ramming it down the theme developers’ throats when it would cause development and usability issues without gain was not particularly smart. I am pretty sure I have seen posts on this forum that have admitted that, in hindsight, the customizer wasn’t good. I would contend that an options panel built using the Settings API should have been allowed to coexist with the customizer. If necessary, more stringent guidelines could be laid out for the Settings API to drive standardization. I am pretty sure that several “marketplace” themes largely steered clear of the customizer, hence retained the perception of being more user-friendly and innovative.

    The common refrain had always been, “Themes should not implement plugin territory features”. But menus, options panels, widgets / widget areas, layouts etc. were never really plugin territory features – it is just that for a variety of reasons theme authors had to build in implementations to overcome WP’s native shortcomings. As things got added to the core, instead of alternative (and innovative) implementations being allowed to coexist, theme developers were arm-twisted to overhaul their code (or have their themes culled from the repo for not complying).

    The changes outlined in this post are welcome for sure, but I will not be holding my breath to see if they drive any meaningful change. I have been down this road quite a few times, and I am sure that the rules will soon change and degenerate into something that will become unsustainable for a whole new army of developers.

    • The customizer is a good example of bad apples ruining the experience for others. The team allowed use of the Settings API and other custom implementations for two years after the customizer. However, there were so many security issues, broken interfaces, and just generally bad coding practices that theme authors were not proving the value of alternative systems. There were those of us on the team who fought to give theme authors a chance (myself included at the time), but in the end, we caved because few lived up to expectations.

      I think you’d be hard-pressed to find anyone on the review team who thought the customizer or its requirement was a bad decision. The bad thing was pretty much the total abandoment of continual improvements of its API and documentation from core WP.

      While I am no longer a member of the team, I have wanted them to go this route since Matt first proposed it a few years back. Open the gates, let theme authors who don’t move along with standards fail or develop for their own circle of users. Let them get blamed for when they inevitably break plugins. Let users downvote them for not internationalizing their themes, creating a11y-ready output, and disregarding user-defined settings. But, also let them flourish and try out new ideas.

    • Thanks for that informative and yet disturbing post. I hope that the future is in fact better than you predict, though given past experience, it seems as though that will take more willingness to hear and address concerns of developers than has been the case previously. Almost never does one size truly fit all.

      • “more willingness to hear and address concerns of developers”

        For this to happen I have to stress that developers must speak up.

        Tweeting “I did not like XXX” is not speaking up.
        Being angry on Post status slack is not speaking up.
        Going to the WPTavern comments section saying “requirements are too difficult”, but choosing to not even reply when the team follows up asking which requirement?, that is not speaking up.

        That kind of failed, one way communication only causes frustration and a lot of stress and it is the kind of communication that places us further apart instead of helping us work to improve things.

        If people do not communicate, if concerns and expectations are not voiced, the team and WordPress leadership has no, zero, chance of hearing and addressing those concerns.

        Actively participating in meetings and suggesting ideas. Reading the blog posts about upcoming changes, replying to the discussion posts when the team asks for feedback. Opening Trac tickets and actively working on and following through on those ideas.
        – That is how you make your expectations and your concerns heard.

        • Caveat emptor: My comments are all based on a pre-2016 viewpoint. After that I just stopped caring about the WPTRT’s rules and actions – it in fact started giving me a sense of perverse satisfaction to see it all fall apart. Prior to 2016 I don’t recall having seen your name in any of the forums, so I can only surmise that you became active as a reviewer much later.

          Tweeting “I did not like XXX” is not speaking up.
          Being angry on Post status slack is not speaking up.
          Going to the WPTavern comments section saying “requirements are too difficult”, but choosing to not even reply when the team follows up asking which requirement?, that is not speaking up.

          There was a mailing list for the WPTRT earlier (http://lists.wordpress.org/pipermail/theme-reviewers/ for archived threads). Commenting there was speaking up. Tell me how this was not a case of developers speaking up.

          Several Trac tickets for theme reviews used to get into animated discussions about reasonableness. Commenting there was/is speaking up.

          Several developers were passionate about being heard earlier. But when the WPTRT started enforcing rules of all sorts, rather than spend an insane amount of energy typing out emails in losing arguments, a much simpler alternative emerged: walking away. And at that point, nobody really cares about which shiny new toy the review team has started using, and they will use Twitter and the comments on WPTavern to vent. I say this having been the developer of probably the first non-default theme to have hit a million downloads on .org, and deciding to walk away when the requirements became too much to keep pace with.

          Look – you can talk about it whichever way you like, but the WPTRT’s handling of requirements have caused tens (and probably hundreds) of developers to walk way in disgust and frustration. Something as simple as the use of the term “responsive” in a theme’s name became grounds for heated arguments. And I won’t even get into something like the Customizer becoming a requirement.

          Actively participating in meetings and suggesting ideas. Reading the blog posts about upcoming changes, replying to the discussion posts when the team asks for feedback. Opening Trac tickets and actively working on and following through on those ideas.
          – That is how you make your expectations and your concerns heard.

          You are making an assumption that the people who stopped contributing did not follow any of these. The problem is that there is a difference between getting “your expectations and your concerns heard” and getting “your expectations and your concerns addressed“. The hundreds who walked away had no issues getting themselves heard, rather they had issues with the WPTRT’s general attitude bordering on, “Yup, we hear you. We know that it sucks for you, but that is not our problem”.

          Convince those who walked away that the situation has changed via immeasurable actions (dropping everything but licensing, security and standards-compliance requirements) and measurable impacts (reduction in wait times, increase in contributions from new users), and you will have truly made your case.
          – That is how you prove that expectations and concerns are being heard. Otherwise all of this sounds just too preachy.

          • I’m going to address just a few items:

            I can assure you that Carolina has been fighting for both developers and users for far longer than you seem to think she’s been involved. She’s been down in the trenches doing all the tough work that so many others have not even so much as glimpsed at. Sure, she and I have disagreed on many occasions, but she’s always had the best interests of the project in mind.

            As for the WPTRT mailing list, which we had pre-Slack, participation from developers was still relatively low compared to what was needed to improve the directory. And, much of that participation broke the universal “don’t be a ****” rule. People seemed to be far more willing to break that rule through a fairly obscure mailing list but not on more public Slack and Make blogs. It was an untenable mess.

            On the “responsive” name, you glossed over a much larger trademark/branding discussion, which is something directory personnel are pretty much required to deal with.

            Since you brought it up for a second time in the comments without fully representing the situation, I’ll address the Suffusion theme issue. You threw in the towel when it was taken down from the directory because of a fatal error. You avoided addressing valid licensing, security, and standards-compliance issues the reviewer brought up. The reviewer even offered to help you address things along the way. Then, you misrepresented your theme being taken down to your theme users as simply a reviewer requiring the use of the customizer, which was only a small piece of review.

            The axe finally fell when a theme reviewer pulled Suffusion from the repository and listed Customizer integration as a requirement to get it relisted. And that is when I retired the theme.

            Maybe that was your reasoning, but none of the other items were listed in your explanation. Were there some parts of the review that maybe were not 100% necessary for a functioning theme? Sure. But, those were not the major issues.

            I hope that you at least addressed those security issues for end-users who continued using your theme via GitHub. At the end of the day, folks can quibble over things like using the customizer or branding guidelines, but security should be a top priority for the team.

  8. I’ve used many lite themes that were very nice themes and not obnoxious about upselling, though that is also less common now, unfortunately. I think it would be easier to improve the behavior of commercial themes if WordPress wasn’t antagonistic to the very idea of making money. For instance, there still isn’t a good notice system that allows plugins or themes to get information to users but then forbids them from harassing users excessively.

    It takes financial privilege to donate lots of your time. Allowing people to make money off contributions doesn’t just enable profiteering, it also enables more people to contribute.

    • I think you make some good points. Upselling should be allowed IMO. If you’re going to donate something to the theme directory, that seems fair within reason.

      One thing I’ve noticed is that some of these listed themes are very unclear in terms of what you’re actually getting. They’ll mention certain features but it’s not always obvious if they are included in the “free/lite” version.

      Just in my own research, I find that I have to actually install a theme before I can verify what’s included and what requires an extra purchase.

      That’s where I think my biggest issue lies. It comes off as a deceptive practice, even if the developer didn’t intend it that way.


Subscribe Via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.