Last week, Dan Cameron, creator of Sprout Invoices, received an email from the WordPress.org plugin review team stating that his plugin was in violation of the repository guidelines. Sprout Invoices was promptly removed from the directory and all of its 5-star reviews were also removed.
Cameron had been discounting the professional license of his plugin for customers who gave a review on WordPress.org. In a post expressing his frustration with the way the situation was handled, Cameron said he “figured it was alright to compensate their time” and that customers were free to leave a good or bad review.
The official plugin directory guidelines do not explicitly prohibit compensated reviews, which is why Cameron said he was unaware this was an issue. During his conversation with the plugin review team, he was referred to an article posted on the make/plugins blog regarding the issue:
“If it’s not clear enough, we’re serious,” the email stated. “We even posted on make/plugins.”
At the end of May, WordPress plugin review team member Mika Epstein posted a reminder to plugin developers about not compensating for reviews, as the problem pops up on a regular basis.
“We do not allow for compensated reviews to be on our site, by any means whatsoever, and consider those reviews to be disingenuous,” she said. “While you may not consider getting a product free (or at a discount) to be compensation, we do. It messes up the system, which really is meant for people who legitimately use a plugin to leave a review of their experience.”
However, many plugin developers are not following the Make blogs and look to the guidelines for WordPress.org’s official stance on what is not acceptable. Dan Cameron’s situation has highlighted the issue once again and a post on the Advanced WordPress Facebook group has developers calling for it to be added to the guidelines. The post has accumulated several hundred replies discussing the intricacies of enforcing consequences for incentivizing reviews.
When I asked Samuel “Otto” Wood, author of the guidelines, whether or not there are plans to update them to offer clarity on this recurring issue, he said, “This is not an unwritten rule in the first place. See guidelines 9 and 18.” These include:
#9: The plugin must not do anything illegal, or be morally offensive. That’s subjective, we know. Still, if we don’t like it for any reason, it’s gone. This includes spam, for whatever definition of spam we want to use.
#18: We reserve the right to alter this list in the future. We reserve the right to arbitrarily disable or remove any plugin for any reason whatsoever. Basically, this is our repository, and we will attempt to maintain a standard of conduct and code quality. We may not always succeed, but that is our goal, and we will do whatever we feel is necessary in furtherance of that goal.
For many involved in the discussion, the issue is not whether incentivized reviews should be permitted but rather what is appropriate when it comes to enforcing guidelines that are open to a broad interpretation. Developer Zach Stepek summarizes why the vague guidelines have made the handling of this matter a point of controversy:
What I don’t get is why it is so hard to codify what you are asking people to adhere to. The vague guidelines being referenced in number 9 do not specifically disallow what occurred, especially if the plugin developer doesn’t feel he has done anything morally wrong. And number 18 does little to provide any mechanism of trust in the repository or its keepers. There should be a formal arbitration process that is adhered to before any action is taken, unless the plugin is blatantly egregious in its violation of the guidelines or clearly malicious. This was neither of those things, and the author responded and rectified the situation as soon as he was notified that it was in violation of your interpretation of the guidelines you wrote.
It’s the expediency with which a decision was made to enforce an unwritten requirement, a decision that seems to have little recourse based on your public responses to the community outrage you’ve seen here, that I’m taking issue with, personally. I was planning on expanding my business to include plugins in the near future. I now know not to use the plugin repo in any way, since I cannot trust that an unwritten rule won’t rear its ugly head in the future.
Because the policy was undocumented in the official guidelines and enforced in such a way that it might affect a developer’s business, many commenting on the thread took issue with the swift, unprecedented consequences of removing all the 5-star reviews. Others agreed with the action, citing the FTC’s endorsement guidelines which require reviewers to disclose any compensation or incentives received. Because Cameron did not wait to reward the discount until after the reviewers disclosed the incentive they received, users cannot accurately decide how much stock to place in those reviews.
The Detailed Plugin Guidelines Need to Be Updated
Cameron said one of the reasons he wrote his post was “to make a public precedence so that others consider how they promote/ask-for reviews.” Without a clear policy to help prevent these kinds of infractions, it’s bound to happen again.
It’s understandable that the plugin review team may not want to spell out all sorts of forbidden ways to game the system, but incentivizing reviews is a common violation that has already warranted a post on the make/plugins blog. If an issue generates this much debate and controversy, it’s time to reconsider how well the current guidelines are serving the community.
Although the title of the document is “Detailed Plugin Guidelines,” several of them are anything but detailed. The two referenced regarding this issue (#9 and #18) are both subjective and arbitrary. They include vague language, indecipherable expectations, and require the plugin developer to be familiar with the worldview of the document’s author in order to understand what is “morally offensive.”
The language of the guidelines is also unwelcoming and authoritarian, more reminiscent of a childhood secret club than an open source project that powers 26% of the web:
The plugin must not do anything illegal, or be morally offensive. That’s subjective, we know. Still, if we don’t like it for any reason, it’s gone.
We reserve the right to arbitrarily disable or remove any plugin for any reason whatsoever. Basically, this is our repository, and we will attempt to maintain a standard of conduct and code quality.
WordPress is available in more than 150 different languages. Not all plugin authors share the same morals, culture, or exposure to review systems and their effects on directories and marketplaces. A policy of no incentivized reviews is one worth spelling out in plain terms for those who don’t speak English as their first language and who may not regularly read the make/plugins blog.
When important guidelines are not communicated and arbitrary consequences are enforced, it says to the plugin developer: “You’re obviously not from around here, or you would know better.” That kind of small town attitude is unwelcoming to those coming from other software communities where expectations may be different.
Clear-cut guidelines will become even more important when the plugin review team starts expanding to include more members. With the current vague guidelines, new members will need to consult the original review team in order to know how to proceed. It removes team members’ autonomy and authority and ultimately slows down the review process.
The success of the WordPress plugin ecosystem, which includes both free and commercial products, is one of the reasons the platform continues to grow. The general consensus is that incentivizing reviews is unacceptable, but a simply stated guideline would justify a prescribed set of consequences and prevent debates like these from blowing up. Clear expectations presented in language that is easy to understand would better serve WordPress’ global user base and its diverse community of plugin developers.
Excellent summary and could not agree more about your proposed changes. I was following the discussion on Facebook and it was interesting to read the opinions from everyone, but in the end I feel this stuff can be easily prevented by overhauling the incredible vague guidelines.
I think Dan got punished too hard for this, and I hope the core plugin team (or just Samuel) will reconsider his decision.