Cancel

Jeff Chandler

Why Some Theme Authors Are Waiting Two Months or More for Their First Review

photo credit: Βethan - cc
photo credit: Βethan – cc

Within the last few weeks, we’ve received emails from readers wanting to know why it’s taking so long for new themes to be reviewed on WordPress.org. Some theme authors are having to wait two months or more for their first review.

Ashley Evans submitted her theme in June and she’s yet to complete the review process. Throughout that time period, both Evans and the reviewer experienced delays in responding to each other. A few months into the review, the reviewer disappeared and Evans was assigned a new reviewer two days ago.

Understandably, the experience has discouraged Evan’s from submitting anymore themes to the directory:

Back in August, I said, ‘Screw it’ and released the theme as a free download on my blog. This process has basically made me vow to stick to adding plugins to the repo and stop adding any more themes.

I’m not blaming the theme review team since I can only imagine how much stuff they have to wade through. It’s just sad that the process has discouraged me from ever doing it again.

On October 9th, Tammie Lister updated the Theme Review queues and identified a number of themes that fell through the cracks. Most of those themes were approved or are still in the review process.

The System is Broken

Members of the WordPress Theme Review team agree that the system is broken. In June, the team published its suggested roadmap to improve multiple facets of the review process. One of the items on the list to help cut down the review queue is the auto-approval of theme updates. However, the team is still hard at work trying to code and implement changes to improve the system.

Help Them Help You

photo credit: Rob Shenk via - cc
photo credit: Rob Shenk via – cc

One of the items high on the team’s to-do list is to put more effort towards education. In order to do that, Justin Tadlock says the team has to free up resources, “We need to free up our biggest resources, which are the team members themselves. However, we can’t free up those people when they’re spending 100% of their time doing reviews.”

The most important thing theme authors can do to speed up the review process is to check that your theme meets the Theme Review Requirements. According to Tadlock, “The majority of themes submitted don’t follow the guidelines which considerably slows down the process. Themes will often have 20-30 issues or more. If we can get to a point to where the majority of submissions only have a few minor issues, we really wouldn’t have a queue.”

Theme authors who test their themes against Theme Unit Test Data and the Theme Check Plugin substantially improve the system for everyone. What the team needs most is help. Tadlock offers three ways contributors can get involved to improve the situation.

  1. Doing reviews.
  2. Tackling Meta Trac tickets related to the theme directory.
  3. Writing tutorials.

Tadlock isn’t sure how to get theme authors to raise the quality of their themes before the initial review, “That’s the sort of feedback I want to see from fellow theme authors. What do we need to do to help them get their themes ready before submission?”

How to Get Involved

The team is always in need of more theme reviewers. Reviewing themes is a great way to learn theme development and what not to do. If you’re interested in reviewing themes, read the following document from the Theme Review Handbook. It explains how to set up a testing environment with an example of a testing workflow.

The Theme Review Team also has a project meeting every Tuesday, November 10, 2015, 1:00 PM EST in the #themereview channel on Slack.

Exercise Patience

Exercising patience is a difficult thing to do if you’ve already waited eight weeks or more for the first review. However, fixing the system is going to take time. If you want to know about the status of your theme and it has an assigned reviewer, you should ask for a status update within the ticket. If your theme doesn’t have an assigned reviewer, you can ask about its status in the Theme Review Team Slack channel with a link to the theme.

Category:
Tags: , ,
45

45 responses to “Why Some Theme Authors Are Waiting Two Months or More for Their First Review”

  1. In defense of the theme team, the theme and plugin review systems are totally different. We’re much more cavalier about what kind of plugins we accept, since plugins are (by their nature) meant to do anything. At plugins, we’re not stringent about ‘best practices’ either and we don’t have ‘plugin requirements’ in the same way. We try to enforce safety and security as best we can, but people are encouraged to reinvent certain wheels.

    Themes need to be way way way more consistent and deliver the same quality of experience. You really can’t compare the two systems.

    Create forum topic from comment

    • Mika, how big is the plugin review theme vs. the Theme review team?

      I agree with Tadlock’s point on following guidelines. I wouldn’t be surprised that a lot of themes submitted break basic guidelines which ends up choking up the queue for genuine good themes which don’t

      Create forum topic from comment

    • @Mika – Absolutely zero reason for you to feel that you have to defend anything. Personally I think it is insane how much you guys (THE WP Team) take on personally. Obviously your workload is divided up into teams within THE WP Team, but still as WP popularity only continues to increase and more and more folks use WordPress then the strain and workload for you guys will only continue to increase. Users are asking for more personal hands-on attention more and more. Something has to give at some point. ;) I posted a general though in another WP Tavern post to maybe farm out things/tasks to a 3rd party source. Maybe the day is coming (or has arrived) where supply and demand for everything WordPress is just too much to handle for you guys in-house and it is time to farm out non-critical/non-essential stuff to a 3rd party to take some of the workload/strain off of you guys so that you can focus on strictly essential/critical WP stuff.

      Create forum topic from comment

      • The problem is the pool of community-driven individuals who do these reviews of plugins and themes (for free mind you) is much smaller then the pool of total users and developers. For example, Mika works for a web hosting company, with the company being nice enough to pay for some of Mika’s time to help out at WordPress.org. She does not “work” for WordPress.org. Neither does anyone in the theme review team – maybe one works for Automattic? So that means until we can “develop” more community-minded folks to help out, we’ll always have this issue. Actually I don’t think its a completely solvable problem due to human nature (self-interest versus community interest and just general priorities of individuals that can change over time). I do think we can reduce the burden of entry though and that will help in encouraging more engagement.

        Create forum topic from comment

  2. Been around for a year in the Slack Channel (#themereview), engaged in arguments about automation, including other possible improvements for the Theme Directory-

    In the last couple of months I’ve realized that, the whole process of the Reviewing Themes on WordPress repository needs rethinking. It’s broken, sadly.

    Create forum topic from comment

  3. With so many WordPress community members having some dev knowledge, I’m really surprised not more are volunteering to pick up a theme to review. If just 50 community members each reviewed just one theme a month, things would drastically improve.

    The actual theme reviews aren’t all that difficult and takes minimal time (maybe 1 hour a week?) but perhaps the hardest part is just getting started. The trac system is confusing and intimidating, and it’s not always clear where to ask for help.

    There’s the theme reviews handbook and Slack channel like mentioned above but I can understand how it would be difficult to simply get started.

    Create forum topic from comment

    • I think you really highlighted the problem Jami. Getting started feels incredibly daunting.

      Theme review is something I’d like to get involved in but I’m feeling like I don’t really know what I’m doing in the process. Even though the instructions are pretty clearly laid out, the Trac system feels daunting. Especially because I didn’t feel like I really ‘clicked’ with it during my own theme review.

      Create forum topic from comment

    • Plugins yes. A couple of us get emails for every single plugin check in (thousands a day, makes the trac firehose look tiny), and we have SVN block the most egregious issues. But we also have a lot of email filters to pick out naughty people.

      We do rely heavily on people reporting issues to plugins(AT)wordpress.org (if you do, please remember to link to the specific plugin :D )

      Create forum topic from comment

    • We actually just suspended a theme recently for something like this. This was reported to us. We’re currently giving the theme author a chance to bring the theme back into compliance.

      One of the methods we plan on utilizing is a “report” button to report such abuse in the future.

      Create forum topic from comment

  5. I have been planning to release a few free WordPress themes at WordPress.org but I confess this time consuming review process scares me off. Now if I think about it, I guess that’s the reason why I end up releasing a plugin.

    If I may be bold enough to ask, why is the theme review team so strict about such standards, don’t you think that’s actually limits the creativity? That said, I respect the set-standards and understand how it helps. The point I really want to make is that what would happen if theme review team will adopt the standard plugin review like Mika stated “We try to enforce safety and security as best we can, but people are encouraged to reinvent certain wheels.”

    Just thinking out loud.

    Create forum topic from comment

    • This was discussed in depth before the theme review team was created. Some of us thought that the review process was far too stringent restrictive and would limit creativity. The decision however was made to create a restrictive environment and that that the loss in creativity was worth it for the benefits of ensuring the theme repository contained a higher quality of code.

      So basically, you are not alone in your thoughts, but there seems to be more people who feel the opposite way.

      Create forum topic from comment

    • I’m actually all in favor of reinventing the wheel. But, in such discussions about how guidelines limit creativity, we need to be talking about specific things. Specifics matter.

      Designing with certain limitations doesn’t always mean less creativity though. Good designers/developers are going to build awesome stuff regardless or because of limitations.

      There’s also another side to this: plugin compatibility. Many things we get pretty strict about because if we’re not, themes simply break plugins. As a plugin author myself, I rarely see a conflict with a repository-hosted theme. I see conflicts almost daily with non-repository themes.

      Present your case to me on Slack. I’ll either tell you how you could perhaps build it within the context of the guidelines or help make your case for a guideline exception to the other admins.

      Create forum topic from comment

  6. I’d like to contribute to the Theme Review team but I’m worried that I may not have enough time to go through the review process. I’ve been developing custom themes for a few years now – can anyone comment on about how much time it takes to completely review a theme? To give a theme a good look through how much time should I look to set aside or is it really variable on the extense of the given theme?

    Create forum topic from comment

  7. Jeff,
    Thanks, that’s very helpful!

    Although I’ve been building themes for quite awhile now, I had not seen the full theme requirement list for wordpress.org. It sounds like good basic practices, and not as many as I’d anticipated. I’m pleased that I’m doing most of them already. :) I had thought that the Themeforest WP requirements were more arcane, but as I take a look at them now, they don’t look too bad, nor is it hugely different from the .org one. Maybe I’m getting smarter. ;)

    I haven’t been using the Theme Unit Test material, but I’ve been helped a lot by using wptest.io, another excellent set of test data.

    Even though I enjoy doing things my own way, I can see the reasons for enforcing standards. The good results of that can be seen by the general quality being a lot higher now! We who have been around for awhile remember the kinds of clunkers you could run into a few years ago.

    Still, that backlog sounds daunting. So if I release my stuff free, I think I’ll just do it on my own site, yet use the standards to help me build it cleanly. That way, I get the benefit of the good advice without the waiting line. :)

    Thanks, Dave

    Create forum topic from comment

  8. Design isn’t taken into consideration at the moment, that’s the big issue. It would bring the reviewing time frame at around 1 month. Just take a look at the newly released themes, it is really frustrating and depressing to see such themes pass through…

    At this stage you need to wait at least 2-3 months to get a theme accepted & live. If for some reasons you are unlucky and get a fake reviewer that period gets at around 4-5 months. Insane…

    Also, already existing themes being submitted by users, with just a few modifications should get deleted on the spot. There is no innovation/learning in that. They just want a freaking link and flood the trac with useless stuff.

    Create forum topic from comment

  9. I just joined the review team and did my first review over the weekend, the tickets are out of control and they need help. For that review it was opened 15 months ago! And the theme developer had used an options framework, so all that work and time only to be told they had to switch over to the customizer and do more work.

    Themes are so important to the end user, as it’s their main interaction with their site and their site’s ultimate identity. I hope the review team can sort out a better way to keep on track, but in the meantime, if you can, lend them some help.

    Create forum topic from comment

  12. I know the review team is working on improving things because I often go into the Slack and watch and read their meetings. It’s good to know they are aware of what is going on. I also know that they work extremely hard to try and maintain and make the whole process better.

    Still, the review process needs additional fine tuning. I brought this up to their attention before but themes are approved only on the basis of code and not design. I would love to see the theme directory focus on design quality as well, something to seriously compete with marketplaces and to offer users with quality themes. The end-user doesn’t care about code (although important), they want themes that look great and offer features.

    I will say though, compared to just a couple years ago, quality theme design has definitely improved when I look at the directory.

    My idea was to have a 3-tier review process:
    1st is to do an initial visual design review.
    2nd is to do a code review.
    3rd is for the admin’s to give final approval.

    The 1st tier would greatly reduce the backlog of themes and offer up faster review times for the ones that get through, perhaps even increase reviewer/author interaction with recommendations. I believe ThemeForest does an initial visual design review and their review times are down to like a week.

    I also believe that to ensure you have quality reviews, you need quality people. Reviewers should somehow be interviewed by the Admins, as though it’s a job they are applying for. To ensure the reviewer will take the role seriously and not walk away from the middle of a theme review, to fully grasp what it means to be a reviewer, to understand that it does require your time, and to let the author know you are there for them.

    I agree that design can be subjective, but reviews should be 50% design and 50% code quality. People with a flare for design and enough experience would be ideal for this review process.

    I’ll be honest, I’ve considered ending new submissions (I currently have 3 themes in the queue), but we will see how the next month goes. I’m also thinking about adding the downloads to my site and provide support from my site as well…I’ve had many people waiting almost 3 months for my themes to go live at .org.

    I know I’ve been asked to be a reviewer, but I’m still out on that decision for the time being.

    Create forum topic from comment

    • Unless we begin rejecting/failing themes, adding an additional design review layer will actually increase the amount of time that themes take to get through. However, it’d push well-designed themes through the queues faster, of course.

      TRT is in the business of working with theme authors to bring their themes up to certain standards. The comparison with ThemeForest doesn’t apply because they don’t do “reviews” in the same sense. They have more of a pass/fail system, whereas TRT covers each specific issue with theme authors.

      The official directory is a different beast. We try to be as all-inclusive as possible. We want first-time theme hackers as well as seasoned developers. That’s why I, among several others on the team, are almost entirely against any sort of design review. It creates a larger barrier to entry for first-timers. There are some objective design issues that we’re willing to go along with, but the biggest battle we must win first deals with the underlying code. A poor design never broke anyone’s entire WP install. However, some poor PHP code has.

      TRT’s position is that we want to get rid of the initial review process altogether. That’s the eventual goal. We want submissions to be automatically approved and set live. Then, we take the time to work with theme authors to fix any potential issues later. That’s where I believe having design feedback would be most helpful.

      The idea is to become less like gatekeepers and more like educators who pop in to help with things that improve the experience for the end users of these themes.

      Create forum topic from comment

    • Instead of focussing on design reviews per se, I think the best designs should be brought to the front of the repository, and traffic driven to them. This doesn’t need to involve long manual reviews. A simple star system from approved and unapproved dot org users would work fine IMO. That way it can be automated, and not create a backlog like the regular theme review process.

      This gives motivation to theme designers to create something amazing looking, but does not block crappy looking junk from being submitted. Also, one persons junk may be another persons Picasso, so I don’t see any point in blocking something purely based on it being an eye sore.

      Create forum topic from comment

  13. Perhaps one way of encouraging new reviewers to get on board and speeding up the process is to offer a priority queue for anyone who completes a review of another theme?

    I think that contributors could be rewarded like this and many theme authors would jump at the chance to get their theme reviewed sooner simply by reviewing someone else’s, especially since the wait times have gotten long.

    It would also encourage them to familiarize themselves with the review process in more detail themselves and so more quickly identify issues in their own submitted themes, reducing the workload on the other reviewers.

    Another similar option, which may help resolve the backlog, is to (also?) offer a paid priority submission (that goes to the reviewer on completion) … I realize this may conflict a little with the “free” philosophy of wordpress, but as a temporary solution it could be tested at least until the backlog is reduced to a manageable size.

    Create forum topic from comment

  14. On a different note, for myself, I probably won’t bother to submit my new theme (even though it’s awesome) because:
    1. I am a primarily a plugin coder and so my approach to theme authoring is improving on standards, as such my code is hardly likely to ever pass the theme check standards. :-/
    2. Coding the customizer (mostly callbacks) for every theme option is way too much of a pain, and I have specifically used theme options framework to avoid it in the first place only to find out when looking to release my theme it has now been made a “requirement”? :-(

    I personally think the customizer requirement should be dropped, as there can be many options in a theme that do not even fit the need for a being a customizer option as they do not affect the visual outcome at all. Anyone know how strict this requirement is or can I just limit to the actual relevant display options and hope to get approved? If so, I might bother to do it and fix any issues for #1 too.

    Create forum topic from comment

  15. Yes, It’s too long process of Theme Review.

    Nobody know ( Theme Author / Theme Reviewers ) the current process of Reviewed Theme.

    E.g.
    – How much reviewed by Theme Reviewer?
    – Which things are Reviewed?
    – What the percentage of Theme Reviewed? (10%, 30%, 50%, 96% etc.)

    If ( Theme Reviewer ) create any `CheckList` or `ToDo` list of known issues with `Canned Response` such as:

    ————————-

    # header.php

    1. Required: Have a valid DOCTYPE declaration and include language_attributes.
    E.g.
    <!DOCTYPE html>
    <html <?php language_attributes(); ?> >

    ..
    ..
    ..

    # footer.php

    ..
    ..
    ..

    Etc.

    ————————-

    Why This? Is this Useful / Useless?

    – I think Its useful for New Reviewer. Lots of things are common in each theme.

    If there is any predefined `ToDo list` / `CheckList` of known issues with `Canned Response` (It will be Google Sheet Or any other) then:

    – It’s useful for Theme Author to fix the listed known issues.
    – It’s useful for Theme Reviewer to updated the status of reviewed items. (YES/NO)
    – It’s also useful for New Reviewer to know which things are reviewed. He/She will cross verify the reviewed items before approved it.

    ————————-

    MOST IMPOTENT: Reviewer could not approve the theme from reviewing just `ToDo list` or `CheckList`.
    – Reviewer need to check each file line by line.
    – Use developer plugins for review.

    Reviewing themes is a great way to learn theme development and what not to do.

    ————————-

    I think (definitely) `Old Theme Reviewers` follow some common `steps` for any new theme review. Don’t know is this write or wrong. Just share the things which I thought.

    ————————-

    Create forum topic from comment

  16. When I first started developing my theme WP-Forge, I decided that I was going to follow the guidelines set forth by the TRT. I also used the wptest.io like Dave Chu mentioned, which is, in my opinion, better than the standard theme test data.

    However, when I finished developing the theme I did not submit it to the TRT. I was afraid to. I was afraid that she wasn’t good enough and that I didn’t do things properly. I feared that I would have to go back and do it all over again. So instead, I offered it for download on my site as well as Github.

    But as time went on, people were asking me if i was going to submit it to the TRT. Eventually I decided that I would and I did. It took about three months to get my first initial review and to my surprise, the changes that were needed were minor and mostly dealt with the escaping of functions (I believe thats what its called, correct me if I’m wrong). I was actually thanked in an email for doing things “the right way” by a member of the TRT, which made me feel great.

    Now I have a theme that is coded to standards and used by a lot of people and I get told all the time its the best theme they have ever used.

    If developers would just try and do things right the first time and maybe ask questions on the TRT Slack channel before they submit, they may find that the changes they may have to make are miniscule. This could reduce the time it takes reviewers to cover a theme.

    But I do like the automated idea. This could speed things up considerably and allow the TRT to work with the developers. Educated developers would mean better built themes.

    Create forum topic from comment

  17. Otto has already created this plugin: https://wordpress.org/plugins/theme-check/ which checks Themes to make sure they are up to theme review standards. So how about making this plugin mandatory and then taking it one step further and have Otto’s plugin do any/all additional automation that is currently being done by human Theme auditors and once Otto’s plugin is run on a Theme and the Theme “passes” then automate that “pass” by notifying a Theme auditor that the Theme has “passed”. Of course a side by side data/code change comparison would also have to be done at final Theme audit to ensure that nothing was changed after the Theme “passed”. Seems like this would be a smart way to go and build on.

    Create forum topic from comment

    • Theme Check is already mandatory. If a theme fails on any required or warning stuff from that, it doesn’t even get uploaded to the system.

      The more automation the better. Contributing to that plugin would be a great way to help both the team and theme authors out.

      There’s only so much such a code can do though. For example, it can’t tell us that a theme is licensed correctly. It can’t distinguish between genuine and spammy links that are hardcoded. It doesn’t know if custom post metadata is related to the visual design vs. holding content. That’s why we still have human reviews (though we’d like to get away from having to do initial reviews before making themes live).

      Create forum topic from comment

  18. This idea just popped into my head and it is either brilliant or just plain dumb. Personally I am leaning more towards the brilliant range. ;) It is fairly common practice in a lot of forums to award “status points”, “status titles”. By adding something like that to all WP user accounts you would be creating incentive to answer other users forum questions. This would probably reduce the WP Team’s workload in regards to answering random forum questions. Personally I feel that I would take on more random WP forum questions if my user account displayed something like this for example:

    Username
    Member
    WP SuperStar (yeah LOL, but it gets the point across)

    Create forum topic from comment

    • forum points don’t accurately reflect someone’s coding abilities though, just how active they might be, unlike the point system on stackoverflow etc which is far more of a combination of ability and activity. but nothing like this is going to indicate how well someone has created a theme itself.

      for themes submissions, a simple user meta flag (or count) indicating the user has a previous theme(s) that passed review could work though, if it would allow them to (somehow?) put new themes online sooner, at least in the trust that they will not do anything dodgy to put their other themes/reputation at risk… and it means they at least familiar with the review process themselves having gone through it before, those themes are more likely to have less problems from the outset.

      and/or, since the entire review process is openly published anyways, there could be an option to just check a box saying “I have self-reviewed my theme and to my knowledge it passes all the required review tests.”
      This in itself could give enough pause to authors to reduce the number of themes submitted that are just plain going to fail as they are not ready yet.

      Create forum topic from comment

  19. I’m glad to see the comments in this thread from the theme and plugin teams. It’s somewhat comforting to know that the Accessibility Team isn’t the only team that’s completely overwhelmed with work, and the next time someone in the wider accessibility community bitches that there aren’t enough accessible themes, or that accessibility-readiness should be required, or that we could enforce plugin accessibility if we really wanted to, I’m passing along this post.

    Create forum topic from comment

  20. We wrote a post on the GoDaddy blog about our experience: https://garage.godaddy.com/webpro/wordpress/how-to-get-your-theme-published-on-wordpress-org/

    Many themes are published in a few weeks or months, but if you’re not proactive with your review, it can take longer. Our approval process took nine months — and some themes have been in the process for more than a year.

    If we learned anything, it’s that we needed to be (gently) pro-active. The review team was overwhelmed and the theme authors who advocated for themselves probably got published faster.

    The Slack channel was a big help … that was by far the most useful source of advice that we found.

    Create forum topic from comment

  21. Just a follow up on this post and I am not complaining – just stating our experience. The last thing I want is to upset the review team and get pushed back in the theme review que.

    12 months on for when this post was published – if the theme review system was considered broken because theme authors where waiting 2 months for the review process – what would you call it now where the wait is 5-6 months for the first review?

    We submitted our Theme Responsi 5 months ago and are still waiting for a reviewer to be appointed (I know they are not appointed – but rather pick a theme to review). At the time we submitted it was number 860 in the review que. Over the last 5 months it has now progressed to number 96 in the que. There are currently 3 themes being reviewed of the 95 theme that are ahead of ours in the que – so that means there are still 92 to be reviewed before ours.

    When we submitted the theme the WordPress version was 4.5.1 – it is now 4.6.1 with 4.7 due out in November – my guess is that it will be released before the theme is reviewed and approved for release.

    We where told on the slack theme review channel to expect a 4-6 month wait for the first review so it is not as if anything has changed in the last 5 months.

    Has anything been done to ‘fix’ the long wait for a theme review that you raise in this post? Again in your post you said some themes where waiting 2 months or more for a full review (oh I wish). Now 12 months on it is 5 to 6 months for a reviewer to even look at the theme for the first time.

    That would suggest that the situation is getting far worse instead of improving.

    Create forum topic from comment

You may also like

Newsletter

Subscribe Via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.