The WordPress Theme Review team has implemented a controversial change to its Trusted Authors Program that puts a hard requirement on participants to join the theme review team and perform a minimum number of reviews in order to continue having their own themes fast tracked through the review process.
“As we can’t figure out a way to bring in new reviewers and maybe keep them on-board after the initial reviews, we decided to make a few changes to the Trusted Authors program,” Alexandru Cosmin said, on behalf of the Theme Review team leadership.
“Trusted Authors will need to review one ticket a month to be able to have their themes set live. Not doing a review doesn’t mean that you’ll lose your privileges or that you’ll have to re-apply. You’ll just not be able to have your themes set live until you finish a review.”
The Trusted Authors program was put in place a year ago with the goal of streamlining the review process for authors who consistently produce high quality code in line with the current guidelines. The idea was to relieve some of the burden for theme reviewers and reduce the queue.
Trusted Authors are required to do a full review of a parent theme (no child themes permitted). Themes that are not approved will not count. After performing the review, the author may then upload a theme and add a comment to it with a link to their latest review that meets the requirement.
The change to the program is controversial, based on the feedback from other members of the Theme Review team who commented on the announcement.
“I understand the reason behind it, but I cannot agree with it,” WordPress theme author Dumitru Brinzan said. “Reviews should be done out of professional desire, not to buy a credit for setting a theme live quicker.
“This might reduce the quality of reviews, because trusted authors are now directly interested in setting more themes live. This means that someone will have to monitor more closely the reviews done by trusted authors. This just feels unnatural somehow.”
Justin Tadlock, a long-time review team member who volunteered as a lead for many years, said he is disappointed to see this idea resurface after he and others shot it down multiple times in the past.
“I assume the team got permission from higher up the chain to run a pay-for-play system,” Tadlock said. “We’ve already established they are not allowed.
“What such systems do is provide an unfair advantage to larger theme businesses with multiple employees. They assign one of their employees to handle a review and keep pumping out themes without missing a beat. All the while, solo developers are forced into ‘volunteering’ with time they may not even have. Not that it’s fair to businesses either; it’s just worse on solo devs.”
Tadlock also said that based on his experience with past incentives, forcing Trusted Authors to join the review team in order reap the benefits of the program will likely result in a decline in the quality of the reviews.
“Making people contribute to the review system should absolutely never happen in any shape or form,” Tadlock said. “It should never be the means in which the team shows favoritism to one author/team over another.
“And, when you tie incentive programs to the review system, you tend to get shit reviews. We’ve already seen this happen.”
Tadlock referenced the Theme Review Incentive program that was implemented in 2014 which became highly controversial due to a number of underlying problems.
“Basically, that program allowed the top reviewers to select the featured themes every month,” Tadlock said. “The original idea (at least from my understanding) would be that they’d select featured themes from the list of themes that they’d reviewed. Instead, they chose their own themes, month after month.
“What ended up happening is that many of those top reviewers would just burn through reviews, focusing on number rather than quality. Bad, sometimes insecure, code would fall through the cracks. Some themes really didn’t even get anywhere near a proper review.”
In response to Tadlock referencing the past incentive program, Cosmin pointed out several differences with the new Trusted Author requirement to join the review team.
“The last time we did this it was a competition for the Featured page (which in my opinion is of higher value than having a theme on Latest),” Cosmin said. “Back then you also had to do a lot of reviews just to get the chance of selecting a featured theme.
“With TAs you don’t lose anything, you either do or not the review, you keep your TA status. One review a month is just 15-30 minutes of reviewing. Either way they are still ‘pumping out themes without missing a beat.’ Any TA author that has time to pump out 3-4 themes a month also has time to do a freaking review.”
Theme Review Team Leadership Did Not Consult the Team Before Implementing Changes to Trusted Authors Program
This change to the Trusted Authors Program seems to have blindsided other members of the Theme Review Team who only learned of it from the announcement today. The idea was not discussed publicly in the #themereview channel on Slack. It was a unilateral decision made by the leadership behind closed doors.
I asked Cosmin for background on the decision and he said it was discussed in a private meeting of Theme Review Team leads that included William Patton and Ganga Kafle. He said the decision just happened while they were discussing the current state of the queue and how things are not going well.
There are 120 themes waiting to be reviewed and Cosmin estimated that authors are waiting approximately two months in order to get their themes approved. He said the changes to the Trusted Authors program are “currently the only viable option with short term results.”
However, Tadlock is concerned that Trusted Authors who didn’t have the desire to review themes prior to the requirement might simply do the minimum possible to stay in the program. It also sets a precedent for requiring volunteer time in order to receive the benefit of a streamlined review.
This particular controversy is another milestone in the Theme Review Team’s perennial struggle with an unmanageable queue. In the past, the team has entertained suggestions about relaxing the submission guidelines and limiting reviews to security concerns, but changes in this direction never seem to materialize. So far the team has had success with limiting authors to submitting one theme at a time. It slows the growth of the directory but makes the work more manageable for the volunteers who often find themselves knee-deep in manual code review without an end in sight.
The new requirement for Trusted Authors to perform reviews in order to have their themes set live may still be up for discussion if other reviewers continue to raise concerns, but comments from the leads indicate that they want to give it a try before scrapping the idea. In response to Tadlock’s concern about the potential impact on the quality of reviews, Cosmin said the leadership will decide based on how the program goes.
“It’s expected that TAs are experienced authors that know the requirements,” Cosmin said. “We’ll monitor this and if it’s the other way around, we’ll decide then. We get shit reviews right now without having any incentives.”
When the previous theme review incentive program was in effect, some theme houses hired contractors to fast-track reviews. I know several of those contractors. They were told by their employees to pass themes through with minimal friction, resulting in a lot of sub-par themes with bad code to be approved. This idea, however well intentioned, opens the door to the same practices re-emerging.