Theme Review Team Leadership Implements Controversial Changes to Trusted Authors Program, Requiring Theme Reviews in Exchange for Making Themes Live

The WordPress Theme Review team has implemented a controversial change to its Trusted Authors Program that puts a hard requirement on participants to join the theme review team and perform a minimum number of reviews in order to continue having their own themes fast tracked through the review process.

“As we can’t figure out a way to bring in new reviewers and maybe keep them on-board after the initial reviews, we decided to make a few changes to the Trusted Authors program,” Alexandru Cosmin said, on behalf of the Theme Review team leadership.

“Trusted Authors will need to review one ticket a month to be able to have their themes set live. Not doing a review doesn’t mean that you’ll lose your privileges or that you’ll have to re-apply. You’ll just not be able to have your themes set live until you finish a review.”

The Trusted Authors program was put in place a year ago with the goal of streamlining the review process for authors who consistently produce high quality code in line with the current guidelines. The idea was to relieve some of the burden for theme reviewers and reduce the queue.

Trusted Authors are required to do a full review of a parent theme (no child themes permitted). Themes that are not approved will not count. After performing the review, the author may then upload a theme and add a comment to it with a link to their latest review that meets the requirement.

The change to the program is controversial, based on the feedback from other members of the Theme Review team who commented on the announcement.

“I understand the reason behind it, but I cannot agree with it,” WordPress theme author Dumitru Brinzan said. “Reviews should be done out of professional desire, not to buy a credit for setting a theme live quicker.

“This might reduce the quality of reviews, because trusted authors are now directly interested in setting more themes live. This means that someone will have to monitor more closely the reviews done by trusted authors. This just feels unnatural somehow.”

Justin Tadlock, a long-time review team member who volunteered as a lead for many years, said he is disappointed to see this idea resurface after he and others shot it down multiple times in the past.

“I assume the team got permission from higher up the chain to run a pay-for-play system,” Tadlock said. “We’ve already established they are not allowed.

“What such systems do is provide an unfair advantage to larger theme businesses with multiple employees. They assign one of their employees to handle a review and keep pumping out themes without missing a beat. All the while, solo developers are forced into ‘volunteering’ with time they may not even have. Not that it’s fair to businesses either; it’s just worse on solo devs.”

Tadlock also said that based on his experience with past incentives, forcing Trusted Authors to join the review team in order reap the benefits of the program will likely result in a decline in the quality of the reviews.

“Making people contribute to the review system should absolutely never happen in any shape or form,” Tadlock said. “It should never be the means in which the team shows favoritism to one author/team over another.

“And, when you tie incentive programs to the review system, you tend to get shit reviews. We’ve already seen this happen.”

Tadlock referenced the Theme Review Incentive program that was implemented in 2014 which became highly controversial due to a number of underlying problems.

“Basically, that program allowed the top reviewers to select the featured themes every month,” Tadlock said. “The original idea (at least from my understanding) would be that they’d select featured themes from the list of themes that they’d reviewed. Instead, they chose their own themes, month after month.

“What ended up happening is that many of those top reviewers would just burn through reviews, focusing on number rather than quality. Bad, sometimes insecure, code would fall through the cracks. Some themes really didn’t even get anywhere near a proper review.”

In response to Tadlock referencing the past incentive program, Cosmin pointed out several differences with the new Trusted Author requirement to join the review team.

“The last time we did this it was a competition for the Featured page (which in my opinion is of higher value than having a theme on Latest),” Cosmin said. “Back then you also had to do a lot of reviews just to get the chance of selecting a featured theme.

“With TAs you don’t lose anything, you either do or not the review, you keep your TA status. One review a month is just 15-30 minutes of reviewing. Either way they are still ‘pumping out themes without missing a beat.’ Any TA author that has time to pump out 3-4 themes a month also has time to do a freaking review.”

Theme Review Team Leadership Did Not Consult the Team Before Implementing Changes to Trusted Authors Program

This change to the Trusted Authors Program seems to have blindsided other members of the Theme Review Team who only learned of it from the announcement today. The idea was not discussed publicly in the #themereview channel on Slack. It was a unilateral decision made by the leadership behind closed doors.

I asked Cosmin for background on the decision and he said it was discussed in a private meeting of Theme Review Team leads that included William Patton and Ganga Kafle. He said the decision just happened while they were discussing the current state of the queue and how things are not going well.

There are 120 themes waiting to be reviewed and Cosmin estimated that authors are waiting approximately two months in order to get their themes approved. He said the changes to the Trusted Authors program are “currently the only viable option with short term results.”

However, Tadlock is concerned that Trusted Authors who didn’t have the desire to review themes prior to the requirement might simply do the minimum possible to stay in the program. It also sets a precedent for requiring volunteer time in order to receive the benefit of a streamlined review.

This particular controversy is another milestone in the Theme Review Team’s perennial struggle with an unmanageable queue. In the past, the team has entertained suggestions about relaxing the submission guidelines and limiting reviews to security concerns, but changes in this direction never seem to materialize. So far the team has had success with limiting authors to submitting one theme at a time. It slows the growth of the directory but makes the work more manageable for the volunteers who often find themselves knee-deep in manual code review without an end in sight.

The new requirement for Trusted Authors to perform reviews in order to have their themes set live may still be up for discussion if other reviewers continue to raise concerns, but comments from the leads indicate that they want to give it a try before scrapping the idea. In response to Tadlock’s concern about the potential impact on the quality of reviews, Cosmin said the leadership will decide based on how the program goes.

“It’s expected that TAs are experienced authors that know the requirements,” Cosmin said. “We’ll monitor this and if it’s the other way around, we’ll decide then. We get shit reviews right now without having any incentives.”


9 responses to “Theme Review Team Leadership Implements Controversial Changes to Trusted Authors Program, Requiring Theme Reviews in Exchange for Making Themes Live”

  1. When the previous theme review incentive program was in effect, some theme houses hired contractors to fast-track reviews. I know several of those contractors. They were told by their employees to pass themes through with minimal friction, resulting in a lot of sub-par themes with bad code to be approved. This idea, however well intentioned, opens the door to the same practices re-emerging.

  2. When decisions like this are made, it makes theme authors rethink their support for the repo. In the 8+ years of submitting themes, I’ve seen a lot of insane ideas and rulings take place that makes one frustrated and the head shaking in disbelief.

    One that comes to mind with regards to the TA program is that it used to be a requirement of at least 1 theme submitted, reviewed with less than 3 issues, and live before the author can be approved for the TA program. However, a new author comes along, submits the first theme, passes, goes live, then applies for the TA program, only to get denied because of a sudden (and unknown change) rule change that you now need 2 themes, not 1. The problem is that the TA info posting of requirements did not have this made known.

    Now we see this new requirement that was decided in a private meeting without consulting with “everyone” involved with the review team. That alone is troubling to hear.

    Another possible controversial ruling is this requirement:

    Not-approved tickets don’t count, they can be closed for 3 distinct issues. You’ll need to do a full review, set the ticket live if you have access or approve it.

    If I understand this correctly, if you perform your “1 review” and it does NOT pass and go live, you as a TA, will not have your theme go live either until you have a successful review that goes live.

    There is also the issue of how long it takes to perform a proper and quality review where Cosmin states 15-30 is all that is needed. But goes on to say:

    Any TA author that has time to pump out 3-4 themes a month also has time to do a freaking review.

    To me, that is a whole other problem that relates to TA’s abusing the program, but we can save that topic for another day.

    However, getting back to the review time of 15-30 minutes is not accurate. If you want to do a proper and quality review, it takes a LOT more time. Consider the fact that not all reviewers are Speedy Gonzales and need more time to review, especially if the theme has complicated coding methods or simply a ton of files.

    After your initial auto-scanning is completed, you should really open up every file and do a visual scan of the code–you would be surprised what you find. Then you have to take into account the time you need to properly write up your findings and recommendations in the theme’s ticket. Now you have to wait for the theme author to respond, make changes, resubmit the update, and then you can go back in to continue the review (hopefully everything will be fixed by the author).

    Let’s not forget about the “Final Review” queue as well, which is also growing in size.

    I agree with Justin, this new decision that was made is a pay-for-play.

    I also agree with Justin with this quote:

    What such systems do is provide an unfair advantage to larger theme businesses with multiple employees. They assign one of their employees to handle a review and keep pumping out themes without missing a beat. All the while, solo developers are forced into “volunteering” with time they may not even have. Not that it’s fair to businesses either; it’s just worse on solo devs.

    I know for myself, I work 12-16 hours per day, 7 days/week.

    Ultimately, big decisions like this, or any decision, should really be implemented by a majority vote from “all” those who are involved with the review team. It’s fine when a team lead wants to come up with ideas, anything that helps, but there are some decisions that should not be made by just a select few…or done in private.

    The queue has been a major problem for too many years, and I yet to see a really solid method to tackle it or even ideas where they spend so much time on with discussions, only to see them never implemented or completed. I know it’s not easy, but I remember many years ago, I brought some ideas to the team and a long story short, the response was basically “Whatever, not going to happen”.

  3. I do not agree with the decision.
    The underlying issue is that we cant do this alone.
    Requests for help have been unanswered.

    If you want to pm me on themereview slack (poena) I might be able to help you bring your suggestion forward or explain why it was turned down.

    One of the problems is Trac and that the team can not test every idea for improvements that we have.

    • Why do you have to make theme reviews so complicated while the plugin repo does only the security review. What are you achieving, who are you serving apart from your big egos?

      None of the themes on envato will pass your review, yet there are thousands of people happily using them.

      End users will decide if a theme is good or not.

      TRT May feel it is lifting the weight of the world. what it does is not required and a waste of effort.

  4. The TA programme should be closed down. WordPress is an open source, all user should be treated as same.

    But TA programme provides the most beneficial privileges to the TA authours. The other users are feeling betray of it.

    A normal user should wait for almost 3 months(this happen if no error in theme) to the theme live but TA ticket goes live in 1/4 time.

    I am on WordPress since 2011, If I planned to be a trusted author, I need a hard work of almost 1 year to live 2 themes and 1 ticket under review, even more, condition there.

    Many users disagree with TA author as above comments shows.


Subscribe Via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

%d bloggers like this: