Cancel

Sarah Gooding

A Day in the Life of a WordPress Theme Reviewer

Emil Uzelac
Emil Uzelac, Theme Reviewer Extraordinaire
Emil Uzelac is a freelance WordPress designer and developer who has been reviewing WordPress themes for the past three years. He and other team members have been blazing through theme approvals and setting records this month on wordpress.org. Recently, I had the chance to chat with him about what it’s like to be on the WordPress.org Theme Review Team.

In case you didn’t know, every theme submitted to WordPress.org is required to meet a rigorous set of standards and guidelines. The theme must also gain the approval a highly skilled and dedicated team of experts, known as the Theme Review Team, who guard the doorway of the official WordPress Theme Directory. This is how WordPress.org ensures that the free themes offered are not full of malicious code.

You may recognize Emil Uzelac from his work on the Responsive theme, which became the most popular free theme on WordPress.org in just three months after its release. After about a year of trying to manage a full time support load for his free theme, Responsive was acquired by CyberChimps, a commercial theme shop. An interesting bit of history here is that Emil’s site ThemeID and his Responsive theme were the first case of anyone ever having a theme so popular on WordPress.org that it was eventually purchased.

Just another day reviewing themes
Just another day reviewing themes

Every day Emil sits down at his desk with his mug and headphones and spends a few hours going through tickets for theme reviews. He’s currently a Theme Review Team (TRT) Admin, as well as full-time reviewer and the Team Rep for the next three months.

Given that the team must be fanatical about standards, I assumed that making it in would be quite a long and difficult road. Email says that it’s actually quite a simple process: “My initiation three years ago was pretty similar to what we have now. A trainee requests a ticket and after several reviews gets promoted to what we call ‘full reviewer.’ Thereafter, that person can assign and review tickets on his own.”

Emil generally spends about 15 hours per week reviewing themes. To stay motivated while going through tickets, he likes to listen to electronica via Pandora. But his chief motivation are the users who will eventually be activating these themes.

Users are my main motivation. It’s my goal for users to experience no or fewer issues once the ticket is approved and the theme is available for download in directory. At the same time I am also motivated to educate new theme authors.

You may be curious about what kind of crazy code people try to stuff into free themes. I was surprised to learn that more often than not it’s simply standards that are ignored. I asked Emil what are some of the most common errors that he encounters when a theme is first submitted and he replied:

You can learn a lot about the theme just by opening a couple of parts like header.php and footer.php.

That is where most common errors are. Some of them would be improper scripts and styles inclusion, plugin-territory items such as META tags and the unavoidable stuff such as “SPAM” within credit links.

An Incentive Program to Bring the Review Queue Under Control

As you can imagine, reviewing all the themes that get submitted to WordPress.org is a colossal task. In July the TRT introduced an incentive program to help motivate reviewers to whittle down the queue. Emil believes that this has been a great benefit to the team:

The program works well and it gives authors the opportunity to be selected in the WordPress.org featured theme section. In my opinion WordPress.org is the best absolutely free way of getting that instant promotion!

The winners each month are allowed to select the featured themes and can even select their own themes if they wish. If you’re a WordPress theme developer and want to get your theme featured, consider volunteering your time with the Theme Review Team.

Advice to WordPress.org Theme Authors

After having a very successful free theme on WordPress.org and having served as a dedicated member of TRT for the past several years, Emil Uzelac has some sage words of advice for authors who decide to post a free theme on WordPress.org:

Support is not easy. However, to have a successful theme you must stay on top of it.

Subscribe to all related tags about you, your site and your theme, as well as the notification system by WPORG and Google Alerts for everything else. When this is established the rest will get easier. Have one thing in mind: WPORG users are simple but very fragile at the same time. Write good tutorials and docs to avoid answering the same questions all over again.

One last and very important thing:

Always ask for a rating and review after resolving each question and don’t forget to note that tickets need to be closed as well.

If you’re looking for a place to get involved at wordpress.org, the Theme Review Team could use some more active members. On average it takes about 30-35 minutes to completely review a theme. There are 118 reviewers but only 5-10 that are currently active. The four active admins include Emil, Chip Bennett, Edward Caissie and Mel (esmi). It’s a huge task for these few to manage but very rewarding in the end. Hats off to all of these folks who volunteer their time to make WordPress.org themes safe and standards-compliant. If you’d like to join, visit make.wordpress.org/themes and get signed up.

Category:
Tags: ,
19

19 responses to “A Day in the Life of a WordPress Theme Reviewer”

  4. Thanks for that all you do Emil. I was wondering if you ever got bored, trudging away through themes but it looks like you keep finding ways to motivate yourself. I’m surprised by the fact that you don’t seem to encounter many themes trying to pass base64 encoding anymore. Instead, it’s just theme designers failing to respect standards.

    An interesting bit of history here is that Emil’s site ThemeID and his Responsive theme were the first case of anyone ever having a theme so popular on WordPress.org that it was eventually purchased.

    I wonder what it was about that theme that made it so popular, so fast? Great piece of history though.

    Create forum topic from comment

  5. An interesting perspective. Now let me tell you how the theme review process goes from the perspective of a submitter.

    1. You submit the theme.
    2. You receive an e-mail saying, “Your theme is FAIL because . TICKET CLOSED. Feel free to comment here or blah blah…” (Um, comment where, on the ticket you just closed in my face? Gee, thanks for the invitation to dialogue.)
    3. You change that pathetic little detail and re-submit. A more merciful reviewer re-opens the ticket and asks, “was that the only issue?” The answer: “no, it’s also FAIL because X, Y and Z”. Which on the one hand *they could have told you from the start*, and on the other hand *your previous theme was accepted with the exact same issues*. Did the guidelines change? Are there simply too many guidelines for one reviewer to know them all? Or are they, you know, *being vengeful*?

    So much for the process. But you see, the guidelines themselves are hypocritical. A simple “this theme sponsored by…” line at the end of the README means automatic rejection, while entire paragraphs advertising pro versions and various paid services are perfectly fine. As evidenced by countless themes and plugins in the official directory where the entire README is one big advertisement. Which is kind of a problem when you’re trying to figure out what the damn thing *does*.

    But sure, it must be a lot of work to review themes when countless naive, starry-eyed developers keep submitting to the official directory anyway. My advice? Don’t. Give those people a break. They’re volunteers anyway and don’t owe us anything.

    Too bad about the broken implicit promise.

    Create forum topic from comment

  7. @Felix –

    Out of curiosity, when was the last time you submitted a Theme?

    You receive an e-mail saying, “Your theme is FAIL because . TICKET CLOSED. Feel free to comment here or blah blah…” (Um, comment where, on the ticket you just closed in my face? Gee, thanks for the invitation to dialogue.)

    First: a ticket closure simply means that the Theme in its current form cannot be approved. We still welcome and encourage dialogue in the ticket comments, to help work through any outstanding issues, to help you get your Theme approved.

    Second, unless the Theme has a major issue (such as improper licensing, or a public-facing “sponsored”/spam link), current practice is to keep tickets open for the duration of the review process. When you upload a new revision, it is appended to the open ticket, and the review proceeds in that same ticket.

    You change that pathetic little detail and re-submit. A more merciful reviewer re-opens the ticket and asks, “was that the only issue?” The answer: “no, it’s also FAIL because X, Y and Z”. Which on the one hand *they could have told you from the start*…

    Theme Reviewers are not quality control; Reviewers are generally happy to help developers work through questions or outstanding issues, but at the end of the day, the developer is responsible for ensuring that the Theme meets all Guidelines before submission.

    It’s not as much of an issue now, but at one time, we were so inundated with submissions – and submissions that clearly made little effort to conform to the Guidelines – that Reviewers simply didn’t have time to find every issue with a given Theme before closing as not-approved.

    If that was your previous experience, you should find things much improved now.

    …and on the other hand *your previous theme was accepted with the exact same issues*. Did the guidelines change? Are there simply too many guidelines for one reviewer to know them all?

    Yes, the Guidelines do change. We are looking for continuous improvements, as well as keeping current with new WordPress core releases.

    And Reviewers are humans (and all volunteers, I might add), and can, will, and do make mistakes. Some Reviewers may miss some things that should have been caught, or may flag something that shouldn’t be flagged.

    The best way to resolve those issues? Comment in the ticket, so that we know there are issues or questions.

    …Or are they, you know, *being vengeful*?

    I know of very few such occurrences, and if found, the admins would deal appropriately with them.

    But you see, the guidelines themselves are hypocritical. A simple “this theme sponsored by…” line at the end of the README means automatic rejection, while entire paragraphs advertising pro versions and various paid services are perfectly fine.

    How is that hypocritical? We welcome developers promoting their own commercial work, but per wordpress.org policy, “sponsored” Themes are not accepted. There’s simply too much opportunity (and incentive) for abuse if we were to allow “sponsored” Themes/Plugins.

    But sure, it must be a lot of work to review themes when countless naive, starry-eyed developers keep submitting to the official directory anyway. My advice? Don’t. Give those people a break. They’re volunteers anyway and don’t owe us anything.

    It appears that you last submitted a Theme a year and a half ago. As such, I can understand your frustrations, in part. The Theme Review Team has undergone quite a bit of growing pains over the three-plus years of its existence. The process is now significantly improved, and under much better control.

    Create forum topic from comment

  8. @Jeffro – Thanks @Jeffro

    Getting bored around the themes is going to be pretty darn hard, well at least to me. Chip is another example of that. We don’t know what the boredom is :)

    You stand correct, there’s new “breed” of authors out there and standards are not that hard to follow.

    Responsive, you know bud I did ask myself the same question many times in the past. Is it the name? Sure name is great and it did help I suppose.

    In addition to clean codes, easy customization and adoption I would say that customer support played very important role here and it still does (4.7 out of 5 stars is not bad if you ask me).

    I can’t agree more, theme really is great piece of history and also live proof that you don’t necessarily have to sell the themes to succeed.

    Thanks for the interview guys,
    Emil

    Create forum topic from comment

  10. First: a ticket closure simply means that the Theme in its current form cannot be approved. We still welcome and encourage dialogue in the ticket comments, to help work through any outstanding issues, to help you get your Theme approved.

    Really? You don’t see how closing the ticket immediately is the equivalent to SLAMMING THE DOOR IN MY FACE? Sure, I can knock a second time, but do you really expect me to feel like it? Like, ever again?

    at the end of the day, the developer is responsible for ensuring that the Theme meets all Guidelines before submission.

    You mean those guidelines that might have changed while I was working on my theme? Or for that matter between submitting it and having it reviewed? Those guidelines even reviewers can miss or forget?

    How is that hypocritical?

    Oh? You don’t see how rejecting a ONE-LINE commercial message at the end while accepting REAMS of advertising INSTEAD of a README is hypocritical? How exactly is ONE LOUSY LINK more abusive? Abusive to whom? Because when looking for themes or plugins as a user I sure feel abused by all the BUY OUR PRO VERSION stuff shoved in my face instead of real information. But nooo… let’s ban a simple, discreet acknowledgement of whoever made the theme in question possible. That’s more fair.

    The best way to resolve those issues?

    To not ever bother you with a submission again. It’s that simple. Isn’t that what you need, less pressure on the review team? There you go. You’re welcome.

    Aren’t things better now?

    Create forum topic from comment

  11. @Felix –

    Really? You don’t see how closing the ticket immediately is the equivalent to SLAMMING THE DOOR IN MY FACE? Sure, I can knock a second time, but do you really expect me to feel like it? Like, ever again?

    I totally get how a ticket closure could be interpreted that way, but it’s simply a misunderstanding of the Theme Review Trac workflow. Unlike in core Trac, where a ticket closure means “this won’t get implemented”, a ticket closure in Theme Trac simply means, “there’s more work; please re-submit”.

    And again: ticket closure is now considerably more rare than it was a year and a half ago. At that time, about 80% of tickets were closed as not-approved; that number is now about 20%. The vast majority of tickets are closed as approved, because the Reviewers work with the developers, in-ticket, to get the Theme through the review process all the way to approval.

    You mean those guidelines that might have changed while I was working on my theme? Or for that matter between submitting it and having it reviewed? Those guidelines even reviewers can miss or forget?

    The Guidelines don’t change that frequently, and never have. When the Theme Review Team first formed, the Guidelines changed frequently as we initially codified them. But since that time, all major Guidelines changes only happen concurrently with major WordPress core releases (with a grace period following release to allow developers time to conform to any new Guidelines).

    And if a developer has questions or problems with the Guidelines, we have communication channels in place to resolve them: either in-ticket (for already submitted Themes) or the Theme Reviewers mail-list.

    Oh? You don’t see how rejecting a ONE-LINE commercial message at the end while accepting REAMS of advertising INSTEAD of a README is hypocritical? How exactly is ONE LOUSY LINK more abusive? Abusive to whom? Because when looking for themes or plugins as a user I sure feel abused by all the BUY OUR PRO VERSION stuff shoved in my face instead of real information. But nooo… let’s ban a simple, discreet acknowledgement of whoever made the theme in question possible. That’s more fair.

    Again: the difference is promoting the developer’s own commercial work versus the conflict-of-interest inherent in allowing promotion of “sponsored” work. The WordPress Theme and Plugin directories simply will not and cannot allow “sponsored” work, for what should be obvious reasons.

    To not ever bother you with a submission again. It’s that simple. Isn’t that what you need, less pressure on the review team? There you go. You’re welcome.

    Aren’t things better now?

    No, actually; the Theme Review Team is handling the “pressure” just fine. It’s been a three-year growing and learning process, but we have a good system now – as the approximately 250 newly approved Themes added to the Theme directory, and over 800 existing Themes updated, in the past three months will attest.

    Create forum topic from comment

  13. Hey bud! How about we move this to the actual ticket instead of here?

    To the ticket where it would be off-topic because it’s not my actual submission(s) but the process itself? To the ticket where only one or two reviewers would see it? Shouldn’t this stuff be known to anyone who plans to submit themes?

    Also, for the future references, if something isn’t right please say something about it, don’t wait years!

    Ah, but you’re assuming I still expect or care to have my last submission approved. I don’t. Learned my lesson all too well.

    Create forum topic from comment

  16. Thanks for featuring an interview with Emil :) Quite the unsung hero to those who may not have realised he was behind Responsive. I have been using the theme for sometime now and aside from how straightforward it is to customise the theme, it has been great being able to refer back to Emil’s own helpful support threads when making tweaks, in order to follow best practice.

    By this I mean, he went beyond answering the basic questions and actually took time to think and help out users regarding specific (interesting!) customisations and all replies being available publicly for future reference. Not that I am criticising other models where you have to register to view content, but its rather nice not having to remember yet another login.

    I would say that – since taking over Responsive – imho Cyberchimps has done a great job updating the theme and maintaining support adequate for most users (props to Ulrich, Bryan and Thomas and the other new staff) – the community is friendly and helpful – continuing Emil’s legacy.

    Am glad to hear he has moved onto the ‘next’ level by becoming directly involved with WP theme reviews. Was wondering what he was doing nowadays ;)

    Create forum topic from comment

You may also like

Newsletter

Subscribe Via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

%d bloggers like this: