WordPress Plugin Directory Cracks Down on Incentivized Reviews, Despite Lack of Official Guideline

photo credit: Green Chameleon
photo credit: Green Chameleon

Last week, Dan Cameron, creator of Sprout Invoices, received an email from the WordPress.org plugin review team stating that his plugin was in violation of the repository guidelines. Sprout Invoices was promptly removed from the directory and all of its 5-star reviews were also removed.

Cameron had been discounting the professional license of his plugin for customers who gave a review on WordPress.org. In a post expressing his frustration with the way the situation was handled, Cameron said he “figured it was alright to compensate their time” and that customers were free to leave a good or bad review.

sprout-invoices-discount-for-review

The official plugin directory guidelines do not explicitly prohibit compensated reviews, which is why Cameron said he was unaware this was an issue. During his conversation with the plugin review team, he was referred to an article posted on the make/plugins blog regarding the issue:

“If it’s not clear enough, we’re serious,” the email stated. “We even posted on make/plugins.”

At the end of May, WordPress plugin review team member Mika Epstein posted a reminder to plugin developers about not compensating for reviews, as the problem pops up on a regular basis.

“We do not allow for compensated reviews to be on our site, by any means whatsoever, and consider those reviews to be disingenuous,” she said. “While you may not consider getting a product free (or at a discount) to be compensation, we do. It messes up the system, which really is meant for people who legitimately use a plugin to leave a review of their experience.”

However, many plugin developers are not following the Make blogs and look to the guidelines for WordPress.org’s official stance on what is not acceptable. Dan Cameron’s situation has highlighted the issue once again and a post on the Advanced WordPress Facebook group has developers calling for it to be added to the guidelines. The post has accumulated several hundred replies discussing the intricacies of enforcing consequences for incentivizing reviews.

When I asked Samuel “Otto” Wood, author of the guidelines, whether or not there are plans to update them to offer clarity on this recurring issue, he said, “This is not an unwritten rule in the first place. See guidelines 9 and 18.” These include:

#9: The plugin must not do anything illegal, or be morally offensive. That’s subjective, we know. Still, if we don’t like it for any reason, it’s gone. This includes spam, for whatever definition of spam we want to use.

#18: We reserve the right to alter this list in the future. We reserve the right to arbitrarily disable or remove any plugin for any reason whatsoever. Basically, this is our repository, and we will attempt to maintain a standard of conduct and code quality. We may not always succeed, but that is our goal, and we will do whatever we feel is necessary in furtherance of that goal.

For many involved in the discussion, the issue is not whether incentivized reviews should be permitted but rather what is appropriate when it comes to enforcing guidelines that are open to a broad interpretation. Developer Zach Stepek summarizes why the vague guidelines have made the handling of this matter a point of controversy:

What I don’t get is why it is so hard to codify what you are asking people to adhere to. The vague guidelines being referenced in number 9 do not specifically disallow what occurred, especially if the plugin developer doesn’t feel he has done anything morally wrong. And number 18 does little to provide any mechanism of trust in the repository or its keepers. There should be a formal arbitration process that is adhered to before any action is taken, unless the plugin is blatantly egregious in its violation of the guidelines or clearly malicious. This was neither of those things, and the author responded and rectified the situation as soon as he was notified that it was in violation of your interpretation of the guidelines you wrote.

It’s the expediency with which a decision was made to enforce an unwritten requirement, a decision that seems to have little recourse based on your public responses to the community outrage you’ve seen here, that I’m taking issue with, personally. I was planning on expanding my business to include plugins in the near future. I now know not to use the plugin repo in any way, since I cannot trust that an unwritten rule won’t rear its ugly head in the future.

Because the policy was undocumented in the official guidelines and enforced in such a way that it might affect a developer’s business, many commenting on the thread took issue with the swift, unprecedented consequences of removing all the 5-star reviews. Others agreed with the action, citing the FTC’s endorsement guidelines which require reviewers to disclose any compensation or incentives received. Because Cameron did not wait to reward the discount until after the reviewers disclosed the incentive they received, users cannot accurately decide how much stock to place in those reviews.

The Detailed Plugin Guidelines Need to Be Updated

Cameron said one of the reasons he wrote his post was “to make a public precedence so that others consider how they promote/ask-for reviews.” Without a clear policy to help prevent these kinds of infractions, it’s bound to happen again.

It’s understandable that the plugin review team may not want to spell out all sorts of forbidden ways to game the system, but incentivizing reviews is a common violation that has already warranted a post on the make/plugins blog. If an issue generates this much debate and controversy, it’s time to reconsider how well the current guidelines are serving the community.

Although the title of the document is “Detailed Plugin Guidelines,” several of them are anything but detailed. The two referenced regarding this issue (#9 and #18) are both subjective and arbitrary. They include vague language, indecipherable expectations, and require the plugin developer to be familiar with the worldview of the document’s author in order to understand what is “morally offensive.”

The language of the guidelines is also unwelcoming and authoritarian, more reminiscent of a childhood secret club than an open source project that powers 26% of the web:

The plugin must not do anything illegal, or be morally offensive. That’s subjective, we know. Still, if we don’t like it for any reason, it’s gone.

We reserve the right to arbitrarily disable or remove any plugin for any reason whatsoever. Basically, this is our repository, and we will attempt to maintain a standard of conduct and code quality.

WordPress is available in more than 150 different languages. Not all plugin authors share the same morals, culture, or exposure to review systems and their effects on directories and marketplaces. A policy of no incentivized reviews is one worth spelling out in plain terms for those who don’t speak English as their first language and who may not regularly read the make/plugins blog.

When important guidelines are not communicated and arbitrary consequences are enforced, it says to the plugin developer: “You’re obviously not from around here, or you would know better.” That kind of small town attitude is unwelcoming to those coming from other software communities where expectations may be different.

Clear-cut guidelines will become even more important when the plugin review team starts expanding to include more members. With the current vague guidelines, new members will need to consult the original review team in order to know how to proceed. It removes team members’ autonomy and authority and ultimately slows down the review process.

The success of the WordPress plugin ecosystem, which includes both free and commercial products, is one of the reasons the platform continues to grow. The general consensus is that incentivizing reviews is unacceptable, but a simply stated guideline would justify a prescribed set of consequences and prevent debates like these from blowing up. Clear expectations presented in language that is easy to understand would better serve WordPress’ global user base and its diverse community of plugin developers.

55 Comments


  1. Excellent summary and could not agree more about your proposed changes. I was following the discussion on Facebook and it was interesting to read the opinions from everyone, but in the end I feel this stuff can be easily prevented by overhauling the incredible vague guidelines.

    I think Dan got punished too hard for this, and I hope the core plugin team (or just Samuel) will reconsider his decision.

    Report


  2. The team at Sprout Invoices is arguably one of the better companies in the market. It’s unfortunate that this happened to their product/team. That said, it’s unfortunate that the plugin team has to bear the burden of policing spam and malicious business practices. There’s a lot on their plate and anything that pulls attention from allowing innovative & secure code safe passage to the public repo isn’t good for the overall ecosystem.

    Because I can’t go a day without calling attention to the theme side of the house in these scenarios, gaming downloads/active themes is just as rampant as any other grey hat practices seen on the plugin side. I’d like to see more of a ban hammer and consistency with guidelines there as well.

    “The success of the WordPress plugin ecosystem, including both free and commercial products, is one of the reasons the platform continues to grow. ”

    This.

    From a product creators perspective, I see a consistent distaste for *commercial* in the repo. Spend any time in the theme review channel and you come across plenty of sour comments about upsells and “crippleware” in theme option pages.

    Don’t get me wrong, I hate nagging ads as much as the next person, but I hail as a business owner looking to make an honest living, and we cannot afford to go without.

    Moreover, it’s not the guideline that bothers me as much as the inconsistency on how they are enforced. That would add to better serving WordPress’, too.

    At the end of the day I realize (and I hope my colleagues do as well) that .org is *not* about commercial products — period. Yet somehow we all know there’s plenty of money exchanging hands through one of the world’s most popular websites. Over time, admins/reviewers/users/product creators met in the middle and loosely shaken hands on the matter, favoring neither “free” nor “paid.”

    While I wish that some commercial “nod” comes our way — perhaps in the form of a badge or verified account, making .org the beacon of marketplaces — I know that is a pipe dream. At the end of the day, this serves as a lesson, to both camps, that we tread carefully in platforms we do not control.

    //

    Great article, Sarah. Thanks for sharing your view.

    Report


    1. This is a great response. Maybe one of the most well-reasoned, level-headed responses I’ve ever read in the Tavern comments actually. Thank you.

      Report


    2. if your theme/plugin has ads, I’ll just edit them out.
      I also have uBlock Origin.

      Not accusing you of having themes/plugins with ads.

      Report


    3. I am glad I am not alone to see that spamming download count and reviews is a big problem in the repository, and I too would like to see plugins and themes getting banned. Reviews and download count/active installs are very important when users pick their plugins, and it is also used to sort the search results I believe.
      There should be a Wall of Shame to show people that wordpress.org will not tolerate it.

      In my opinion, the plugin “WPSSO” games the download count, just look at the irregularities, and look at the author’s other plugins. “Custom Sidebars”, a widely used plugin that had mediocre reviews suddenly got a 100% increase, all 5 star reviews in less than 2 weeks. Fishy right? And the list goes on.
      Again, this is my opinion, nothing has been proved.

      I have seen several websites where you can buy reviews, downloads for your plugins on wordpress.org, just like you can do with facebook likes, youtube views etc. It has to stop to maintain the credibility of the repository.

      Report


  3. Interesting.

    I have had plugin developers tell me how they have their friends from around the world write fake reviews. I have stopped supporting those developers for that reason. Fake reviews are a pet peeve of mine.

    However, given the laxness of applying other rules which are clearly written for plugin developers (especially regarding using WordPress.org Repository and WordPress admin pages for selling), I am amused by this harsh reaction.

    I no longer report any plugins because of (what I believe to be) the arbitrary approach to enforcement by the plugin review team. I just stop using those plugins that clearly do violate the rules.

    The last few years have brought a much more confusing interpretation of what the rules really mean, IMHO.

    This coming from a plugin contributor and someone who’s username has been moderated since WordPress version 3.7. It does hinder attempts to help users in the support forums. I love WordPress. :)

    Report


  4. Why not just tell the party involved in the transgression that it’s not permitted, pull the reviews, and give them a chance to fix it?

    I have pet peeves about gaming the system too, but that doesn’t mean I should execute the nuclear option if there’s a more amicable way to resolve it. You have rules so you can have a game, not the other way round.

    Report


    1. That’s actually what they did. They told the plugin author it wasn’t permitted, pulled the reviews, and the plugin author fixed it.

      The nuclear option (which has to be used on occasion but not here) is to remove the plugin altogether, pull the reviews, and ban the plugin author.

      Report


      1. I guess I was thrown off then by

        Sprout Invoices was promptly removed from the directory

        . Did I misinterpret that?

        Report


      2. It was probably just temporarily removed until the code was updated. We sometimes have to do this on the theme directory too. The plugin is back up now. It wasn’t permanently removed.

        Report


      3. Thanks Justin. I respect the plugin team and know it’s not an easy job, especially with so few to oversee them. Read more harshly than it was.
        Best, JT.

        Report


  5. Sarah, you’ve really hit the nail on the head here:

    When important guidelines are not communicated and arbitrary consequences are enforced….that kind of small town attitude is unwelcoming to those coming from other software communities where expectations may be different.

    What’s particularly disturbing is there are one set of rules for friends and sponsors (hello Yoast and JetPack) and others for strangers and/or competitors. The arbitrariness and vagueness of the rules are essential for allowing inconsistent judgements.

    Alas, this kind of ethical rot permeates all levels of Western society now. The parallel to the Clinton email server fiasco is striking. There is one set of rules for Hillary Clinton’s violations of secrecy laws vs David Petreus vs Edward Snowden (or any of the other good faith whistleblowers).

    I suppose that under such circumstances expecting moral probity or consistency from the plugin review team would be too much to ask. It’s difficult to criticize them too fiercely (a defence they are quick to call themselves) as the majority are unpaid volunteers. What would help is to shine a bright light on the plugin team’s activities and decisions.

    Your post is a good first step. Like the public courts, the plugin review system should be:

    1. clear guidelines
    2. a list of decisions with reasoning, including ALL incidents and all decisions

    When decisions are public and documented, it will be much more difficult for insider interests to put pressure on “friends” to render inconsistent judgements.

    Report


    1. What’s particularly disturbing is there are one set of rules for friends and sponsors (hello Yoast and JetPack) and others for strangers and/or competitors. The arbitrariness and vagueness of the rules are essential for allowing inconsistent judgements.

      This!

      Report


    2. Once again, unsubstantiated claims of favoritism. It’s all a big conspiracy, man.

      Never mind that none of the people on the plugin review team work for Automattic (or Yoast, for that matter). It sure would be nice if you could back up your claims with what you actually know instead of what you think.

      Report


      1. Yoast breaks every ethical convention in polluting the backend admin interface of WordPress. His update/upgrade notifications go to Subscriber level users half the time. Yoast has completely polluted the admin interface to the point there are three plugins dedicated only to removing Yoast spamvertising. The fiasco of foisting a broken beta with removed free functionality (moved to pro) in v3 would have earned another author a trip to the doghouse and a forced fork (for removing free functionality): Remove Branding for Yoast SEO, SO Hide SEO Bloat and Remove Yoast Notices. On uninstall, Yoast leaves huge detritus in the database against WordPress conventions and requires a third party plugin for proper removal. Yoast freely reviews his own plugins with five star reviews, facing neither penalty nor warning.

        Here’s what the very sensible voldemarr wrote about Yoast plugins:

        Honestly, I’m just sick of going into my wp-admin and seeing “YOAST!!!!!” all over the place.

        Yes, WPMUdev is just as bad. But they have been banned from the WordPress.org plugin repository. Yoast SEO is not a free plugin. It’s a come on for a $259/per domain package of mainly unnecessary bloatware, the useful features of which should be included in the main plugin. Anyone else purveying crippleware with such odious manners would long ago have been sent to the nether lands to rot. His selfish behaviour hurts WordPress end users while it damages both WordPress’s reputation and that of whole pro plugin ecosphere.

        I’ve seen first hand evidence of Yoast and JetPack’s involvement in their own reviews, with plugin review team members and .org moderators running to Yoast or the JetPack team with hot issues to resolve as a team. Others might have seen their plugin pulled for similar issues. There is a huge amount of crossover between Automattic (JetPack) and Yoast and the plugin review team. Not conspiracy theories but grim reality.

        To achieve anything like fairness, the WordPress community desperately needs more transparency in the guidelines and with the decisions (published with reasoned supporting arguments, including actions and redress).

        Report


      2. There is no rule about reviewing your own plugins. I always review my own plugins if I think they’re good.

        Report


      3. Ryan, the issue is not about any one transgression (the digression into the particularities of Yoast was forced by Drew’s “conspiracy theories” charge, it would be better to focus on the big picture). The issue is one of vague guidelines inconsistently enforced with a lack of transparency about the process. It is time for clear and explicit guidelines, consistently enforced.

        Issues of due process, equality before the law and nepotism are not unique to WordPress. The Magna Carta solved similar issues for feudal England.

        Report


      4. I agree Yoast is invading too much of the admin UI.

        Report


      5. Unsubstantiated?

        I was harassed by one of the core committers and no sanctions were placed on this person. Luckily, he was friends with Nacin and a core committer. All he had to do was delete his tweets and all evidence was gone. I still have the tweets I sent and what Nacin wrote me. If you want evidence of nepotism, then let me provide it to you.

        When I say harassment, I am not talking about the usage the core team uses, where if you don’t like what a person is saying or how they are saying it, then it is harassment. I am talking about the legal definition of the term harassment. I am talking about had I saved the tweets and shown them to the police in Missouri, then they could have taken action. Seeing as Missouri has a law against that type and level of abuse. I doubt that if you had shown my tweets, they would have the same level of concern.

        So yeah, what happened to this person? He was made a lead of WordPress. Yeah for justice! Either a rule is applied consistently or you will suffer the people crying foul.

        What I am saying is that when you live in an echo chamber, it is difficult to understand what those on the outside are talking about. What? Are you seriously going to harm your friend? Come on. I would be more surprised if you did apply rules consistently, but the reason for vague rules is so that you are able to apply them inconsistently. You can just say that fair is fair.

        I find that there are many in the core team that like to keep things as vague as possible. I think it is disrespectful to the community. If the review team respected the plugin and theme developer community, then their concern would be to educate. The idea that someone has to go out of their way to look up guidelines on an external site to be a conspiracy to apply rules based on whim. To be a king, instead of a leader.

        I think I am biased. Usually, I assume good intentions, but I find that too many of the core team uses logical fallacies for it not to be a pattern. I’m not sure if it is a consequence of what appears to be an echo chamber some of you appear to be in. I find the general lack of acceptance of criticism beyond that which is vague to be worrying.

        I find the level of censorship worrying, but well it is smart. If no one knows about criticism, then the people complaining are part of some conspiracy. If no one understands the level of nepotism in the WordPress community, then there is no nepotism, right?

        WordPress leadership is a neat case study. I made some predictions. Some have come true at a faster rate than I predicted. I had given it 2 to 3 years before more aggressive censorship would take place, but any fool could have realized that would be happening based on the positions of the core team and I guess those in the friend circle.

        My second prediction is unlikely to happen. The backlash for WordPress is too sparse and disorganized for any mass exodus to occur in any near future. I predict that if there was a viable fork of WordPress, that you would see a steady stream of people drop off and transition communities. The condition for the fork, would be that it be well supported.

        Report


  6. As much as I agree with that the guidelines should be more clear…just because you can do something, doesn’t mean you should do it. Show integrity and don’t do it.

    So many reviews are from 3 categories

    1) The author him/herself
    2) family/friends of author/business
    3) competition writing negative reviews.

    This applies not just for WordPress themes/plugins.

    I just did a few review on at TripAdvisor. There was a paragraph that obviously I agreed to that says I can’t review a place if I am related (if any way) to the place owners or If I am been given an incentive to review.

    No one has ever written or said specifically that I can’t shoot Sarah Gooding. I AM NOT GOING TO. Because morally I know that is wrong.

    Plugin/Theme Authors could show some integrity and not milk the system.

    I AM NOT SAYING ALL PLUGIN/THEME AUTHORS MILK THE SYSTEM.

    Report


    1. just because you can do something, doesn’t mean you should do it. Show integrity and don’t do it.

      Exactly!

      Report


    2. No one has ever written or said specifically that I can’t shoot Sarah Gooding.

      What a strange thing to say.

      Report


  7. Read the post on Tavern, read Dan’s post, read Julien’s post, read the comments and discussion in AWP group.
    More people tend to support the developer and make a valid complain about the way .Org team has laid out their rules. We need improvements in rules to better convey the freedom and limitations.
    Let’s build a “community” not a “Kingdom”. Sigh!

    Report


  8. In India, during elections, a lot of candidates distribute money, drugs, alcohol and other material incentives (bribe/incentive/discount, same thing) to make them go and vote.

    They could go and still vote for someone else. However, it still is unethical and even illegal now.

    If there are two competitors, one who incentivises voters to go and vote (even if theoretically they could vote negatively) and the other doesn’t, the first has a clear advantage.

    Second, the legitimacy of all votes become zero, irrespective of whether they were genuine votes. Whether the candidate has great credentials or personality or past work history doesn’t mean anything here.

    This feels like the same to me.

    Report


    1. I think this is more related to common sense and basic judgement of what’s fair than a question of guidelines:

      When important guidelines are not communicated and arbitrary consequences are enforced….that kind of small town attitude is unwelcoming to those coming from other software communities where expectations may be different.

      The current attitude among a lot of peers and some people here seems to be that unless something is explicitly disallowed, it is fine to do it. Reviews due to their nature and utility need to be fair and unbiased (a discount is a clear bias).

      I think the review team is changing that and it’s a welcome change. Being one of the largest open source community, if WordPress follows the example of other ecosystems and is lax on the principles of fairness and freedom, it’ll become irrelevant just like them.

      I feel this is a good occasion to get the core message of being ethical and fair, using common sense and on principle to avoid interfering with things that are supposed to be free and fair. It’s an important lesson for all of us.

      Report


    2. The only incentive that the user should have is the awesome experience your product gives them. After that it’s fair competition for your skills as a product developer to give your users such a good or bad experience that they are forced to write a review rather than resorting to tactics (gimmicks/strategies, same thing).

      At the cost of feeling like a plug, I have had a 5-star rating on one of my plugins (Widgets in Menu) because the header image made them feel obligated to do that and write this review:

      I don’t use the plugin, but it deserves a five star rating just for the brilliant header image lol

      It is a little stupid (like a child, not like some orange haired people across the globe) but it says something. All I had to do was pay attention to this part of the user experience and I got a review!

      (Review team,please don’t delete it, please, I love that one more than the others :) )

      Report


    3. If we just make sure the total experience (includes small things like icons and header images and obvious important things like readme, documentation, support, etc) is awesome, we’ll get our fair share of reviews and ratings.

      We need not offer something for users to write reviews, except a great experience. That’s the message I get in all this. Nothing else.

      Report


  9. I was planning on expanding my business to include plugins in the near future. I now know not to use the plugin repo in any way

    If anything else, this kind of reaction justifies the whole case to me. Ever since WooCommerce has generated enormous commercial success being a free plugin in the repo, the notion of “let’s use the repo for marketing” has been far too preconceived.

    Plugins providers who understand and accept the repo’s purpose will hopefully think twice before using it as a distribution channel for their businesses in the future. It simply hasn’t been built for that purpose, and whoever tries to bend it to their will should take responsibility for their own decisions when things go wrong.

    Report


  10. *Raises hand to no one in particular*

    Can we keep an eye on the ball and focus on the topic at hand?

    While it is tempting to make all posts on WP Tavern into a conspiracy love fest about everything that you may personally dislike about WordPress, the forums, Matt, Yoast, Jetpack, etc. the topic is this:

    1. Someone compensated users for reviews.
    2. As a result of that behavior, their 5 star reviews were deleted.
    3. Their plugin was temporarily closed in the plugin repo so the author could deal with the issue and resolve it.
    4. Once resolved the plugin was opened up again on the repo.

    5. And this is the point of the post here, the plugin guidelines don’t say explicitly about this.

    The guidelines have wiggle room and while I myself don’t think gaming the reviews needs to be explicitly stated, many people do.

    Gaming the reviews are wrong. That’s what happened, the author now has an opportunity to learn from it. The community has that opportunity as well and that’s what I think should be the lesson from this event.

    Report


  11. Guys, there’s two versions of the plugin guidelines. One is on wordpress.org, the other one is on developer.wordpress.org.

    The two versions are not identical. The Developer page has more content, including this additional text under point 9:

    This includes (but is not restricted to) the following examples:
    – Keyword stuffing or SEO scamming in the readme.
    – Compensating or blackmailing users for reviews.
    – Creating sockpuppet accounts to generate fake reviews.
    – Taking other developers’ plugins and presenting it as original work.

    Now, that content wasn’t there as of April 21 (the most recent Wayback snapshot), so it could have gotten added after this issue came up. From what I can tell, it looks like the plugin team updated their docs in one place, but not the other.

    I also don’t understand why they need two different pages for these guidelines, but maybe somebody with more context than me can weigh in on that.

    Report


    1. It looks like that guideline was added just yesterday, not sure what time. It’s rather surprising considering the resistance to updating the guidelines that I saw in the discussion on Facebook. There hasn’t been an announcement on it yet.

      Report


  12. Paid reviews is equivalent to sponsored content etc. US and most EU countries have regulation saying that if you got paid to review something you have to notify the reader about it. Payment does not have to be monetary it can be discounts, products etc.

    Report


  13. I’m not really sure what the issue is here. Developer gamed the system. Developer had any benefits gained from gaming the system removed. Developer warned.

    Whether it is or is not explicity specified, I’d have thought this was quite obvious that you can’t go paying people to write reviews.

    Report


    1. Stop being like that. This is the lawyering and entitlement age, either people “deserve” or “it is not against the written rule”. people just forgot (never learned?) how to behave and they are surprised when it has ramifications.

      Report


    2. Ryan, I think there’s the idea that enforcement is selective and that’s just not the case. No matter how much some insist on “nepotism” or conspiracy theories. ;)

      Enforcement happens when it’s found. Some may not like author’s reviewing their own work, some may believe that other reviews are “bad” but that’s not the same thing as someone doing something they shouldn’t have done and getting caught.

      Forum moderators review the reviews. The majority of the time those reviews are fine even when they’re angry, self-reviewed by the plugin author or even about a pro version of the free plugin or theme.

      This is one case where the plugin author did something that was found after the fact. If it came up 2 years ago the result would have been the same and nothing would have been different except for the number of reviews in question.

      The forums are staffed by unpaid volunteers doing their best on their own free time out of the goodness of their hearts. Honest.

      The idea that those volunteers are Automattic Robots (Autobots?) or Yoast Drones (Yoasties?) or whatever is derogatory. It’s not true regardless of whatever conspiracies or unhappiness some like to insist on.

      It’s also not a good idea to hold an “open court” because it’s not enforcement by popularity. This isn’t the first time this has happened and it probably won’t be the last. The idea isn’t to shame people or debate these things; the author did something that’s not permitted. It was dealt with and hopefully a lesson is learned.

      Report


  14. It’s not the plugin team’s job to make sure that plugin devs know and follow FTC rules. That’s the developer’s job. It seems to me well within the .org rights to take something down that’s not following those rules. And, it’s not like it was down forever. The dev was told about the issue, corrected it, and now it’s back up again. Maybe a link to the FTC rules in the plugin guidelines would be appropriate, but they shouldn’t have to spell out things that are already clear there.

    Report


  15. Hi. Let’s clarify things.

    Note, this is a big comment. If you want to reply, then by all means do so, but please, don’t cherry pick from my quotes. I’m not looking for an argument, and I won’t engage in that kind of discussion. I’m just explaining how things are at present. :)

    Reviews and managing them

    First, the reviews system is part of the forums. As such, the Support Forum moderators have power over it. Since we introduced the review system many years ago, the forum moderators have done exactly what they are supposed to do: moderate it. So when they discover reviews that should not be there, they delete them. Then they notify the plugins team about the problem. So in this case, all the affected reviews were removed before the plugins team was told of the issue. Note, *all* of them, not just the five-star ones. There was some miscommunication about that, I believe.

    It is the job of the plugins team to interact with the plugin authors. Therefore, if a plugin is doing something bad, we tell the author to make it stop doing that bad thing. If the plugin author is doing something bad, we ask them to stop doing it. Usually this is all that happens. Everybody gets sorted, everybody gets on with their day. In this particular case, Mika asked him to remove the code, he did, we considered the issue finished.

    In this specific case, I entered the scene after Mika asked me to help. I explained the situation to Dan, as I’ve done many times before. This is not the first time we’ve removed false reviews. It’s not even the tenth time we’ve done it. Reviews get removed for all sorts of reasons. Again, the forum team handles it. They’re quite good at their jobs.

    Now, note that nowhere in this process is there any room for “punishment”. We don’t punish. We’re not judges. We are not seeking justice. We’re simply seeking to fix problems with the directory and the forums and the WordPress.org website. That’s all we do. We work to make the website better, smoother, more accurate, and more useful to the millions of users that depend on it running smoothly.

    So, if you want to talk about this subject, then please think of it from that mentality. Because we definitely do. I appointed most of the forum moderators. I recruited the plugins team. I’ve helped with the theme review team by providing them with much of their tooling over the years. I’ve spent a lot of time and effort in finding the right kind of people and getting them the powers and tools and information they need to run this 13-year-old website. And I’ll back their decisions… Or sometimes reverse them, but only rarely. We have a good team of good people doing good work. :)

    On the topic of “Guidelines”

    I wrote those detailed plugin guidelines 5 years ago (give or take). At the time, I was against it. I still think that the golden rule is good enough. Call me an optimist. But, sometimes people need details. So, since it seemed we needed them, I wrote them, everybody on the team agreed with them (with only minor wording changes), and we adopted them. They have come in handy over the years, mainly to give us a place to point to when somebody asks where “it’s written”. We can say “right there”.

    But, the truth is, I don’t like those guidelines. I don’t like the idea of having “rules” or “regulations” in the first place. I prefer to rely on people. People who make decisions, who take stands, who believe in what they do enough to be able to argue for their points. That’s how we make open source software, isn’t it? Decisions are made by discussion, sometimes argument. People work together to create all this cool stuff, for free, just because they want to. I mean, that’s pretty cool, right? I love it. I love that idea, that concept. I love it when people do that, because they just become the best people when that happens. It’s great to watch.

    At the same time, there are others outside of that “circle”. And I understand that feeling too, that the circle itself seems closed to some. There is a viewpoint that there’s a sort of “groupthink”, and if you disagree with the general principles, then you’re shut off from the community in question. It’s not a good thing, and one we have to actively fight against in all things. You see it in core all the time, they’re constantly fighting that battle. Outside opinions are the best ones. Prevents stagnation and rot. Any healthy open community needs fresh input, constantly. New ideas, new ways of seeing things.

    This is why I don’t like the guidelines. They blockade us into a fixed circle. Not healthy. Not good. But after what we’ve seen in the last 6 years with the plugins review team, I can say this in confidence: They are *necessary*. The directory has a big fat target painted right on it. We’ve been mostly successful in keeping the bad out, the good in. But bad/good is the same as black/white. The real world has shades of gray.. a full spectrum of color in fact. That makes the easy decisions hard, and the hard decisions impossibly difficult.

    That said, the guidelines should be revised. But not just a few things added here or there. Mass revision. Big update. We’re working on that. Have been for a while. Before this post or even this particular discussion.

    Now, the only reason I’m mentioning it here is because when that comes out, because of this story, then it’s going to be seen as reactionary. But it’s not, really. Unfortunate timing, perhaps. The plugin directory revamp has been going on all year. Right along with the idea of expanding the review team. And thus, by necessity, changing things up. Shake up the system. New review process. More reviewers. More feedback to authors, especially new ones. Right now, plugin reviews happen fairly quick because we’re not checking for things like code quality. Would like to, of course, but just don’t have the time. More eyeballs might help that along.

    Conclusion

    Anyway, that’s my thoughts on the topic. I’m still considering the issues surrounding Dan’s specific problem. Two years certainly is a lot of reviews to remove. And it’s not every day where somebody I respect makes this big of a mistake, with such large consequences. And like I said, we aren’t about punishment, in any way. But it’s a complicated issue. And I’m still thinking about it. Dan sent me a list of the reviews that were “paid for”. So, I’m thinking about it.

    Okay? Cool.

    Report


    1. Thanks for clarifying, Otto! I saw that Mika updated the guideline in question yesterday. I understand that you don’t like the idea of having guidelines in general. I think if we’re going to have them at all, the tone should be revised to be more welcoming with clear expectations stated so that nobody is left wondering. It sounds like you wrote them with a knife to your throat. ;) I mean, you can tell you didn’t want to write them in the first place. It would be cool to see them rewritten with a theme of “Let’s all work together to make this the best place on the web to find and host WordPress plugins.” :)

      Report


      1. To be honest, I don’t think the tone is the problem with them. Not everything can be happy and cheerful. I’m not a happy and cheerful person, by nature. :)

        However, what they currently are is very *factual*. Blunt. To the point. Mainly, I was going for “short”. They got revised with details later because people kept being armchair lawyers and nitpicking over interpretations. Whenever somebody nitpicked too much, I went in, added to them, and said “there, better”? That gets annoying after a while.

        We’re not changing them because anybody feels that they’re unfriendly, or because somebody thinks that they’re too non-specific. We’re changing them as part of the entire process of changing how we review. We’re changing them as part of how the new directory is expected (hoped) to work. New directory, new forums, new systems, new processes. All part of the grand work.

        So, might be a while. Still, expect places to discuss the proposals and ideas before too long.

        Report


      2. People like nit picky details in rules. I recommend fighting against that and simplifying them as much as possible.

        If you have a bajillion very specific rules, then people will find ways around them. If you have a few very vague rules that get enforced based on the opinion of the enforcer, people can’t really wrangle out of things and you have a lot less hassle IMO.

        I’m currently going through this process with an entirely unrelated project. People keep trying to add rules and specifics, but I’m always trying to simplify and generalise them, because otherwise we end up with both a complicated set of rules to maintain and a bunch of people looking for ways to wangle their way around them.

        Report


      3. “To be honest, I don’t think the tone is the problem with them.”

        They sound a bit agro to me.

        Report


    2. This discussion made me think of the massive volumes of the tax code and lawyers looking for loopholes. :)

      My comment is that having clear guidelines, however they are structured, is not bad. They serve everyone, not the least of whom are the reviewers and moderators.

      As I have said elsewhere, I’m fine with those paid for reviews being removed. The reviewers / moderators have a tough job.

      Report


  16. The review system for plugins is antiquated. Before an end-user can leave a review for a plugin, they must first register for a WordPress.org forum account.

    It’s 2016. Regular people don’t use forums.

    I have one plugin on the repository. If I relied on the plugin’s official support forum for end-user feedback, I’d know next to nothing to what people using my plugin had issues with. Again, most people don’t have forum accounts. I found it a lot easier to include my email address and eventually my phone number in the plugin if an when people have questions.

    People should be able to leave reviews with their Facebook and Linkedin accounts.

    Report


    1. It’s 2016. Regular people don’t use forums.

      And what is your suggested alternative?

      Report


    2. It’s 2016. Most people don’t have LinkedIn accounts, and most people stopped using Facebook when their parents joined it.

      See, I can make things up too! ;)

      The ability to create an account could be, and probably will be, streamlined. But that doesn’t mean we’re not going to require logins, in some manner.

      Report


    3. This makes 0 sense. I would say the vast majority of people who have a WordPress website, have a WordPress.org account, therefore, the registration/login point doesn’t make any sense in my mind.

      Plenty of people use forums and forum software is getting bigger and better however the focus is shifted. Communities are niche and there are a lot of people but they don’t receive a lot of action via social media they get reoccurring visitors. There are tons of gaming, security, etc forums on the internet.

      If you are putting your phone number in your plugin then no offense it’s not a popular plugin. If we had someone like Yoast or All In One SEO do this they would be swamped with questions.

      Forums do work well when plugins become extremely popular. They are like a ticket system. Your idea sounds great and all but it becomes very hectic to answer questions for users when you have millions of sites using your plugin. Not to mention the community can’t give answers either. Ever since W3TC support went AWOL if there were no forums I think a lot of people would be struggling.

      Report


    4. I think instead of a phone number, you should include a fax number. This is 2016 after all.

      Report


  17. Guidelines would be needed for users too. IMHO too many aweful reviews are left without ever asking for support.

    Report


  18. A wall of shame? My goodness! Based on the draconian style of writing and enforcing of the current ‘guidelines’ I think the last thing we need is a shaming wall. Don’t even suggest it to them. There must always be a path to redemption for anyone that breaks rules. But these aren’t even rules.

    This is the only organization I’ve ever seen where guidelines are even enforceable. Guidelines by their very nature are meant to be suggestions or processes. If these are rules, call them rules and make them clear and concise – not vague and arbitrary – with consistent consequences. I agree with the the author that these sound like rules from a childhood club.

    Report


  19. Just reading through the comments on this. In fairness what Dan did is wrong in many peoples eyes mine included.

    Whilst I do not condone what Dan did, I understand why he did it.

    Reading reviews on WordPress.org can be like pulling teeth.

    By that I mean some of the reviews are quite frankly spammy crap.

    For this let’s take a look at Yoast

    https://wordpress.org/support/view/plugin-reviews/wordpress-seo?filter=5

    Notice that a few on the front page have two words and a five star rating, some are nonsensical.

    Follow the pages through and what you have are some in-depth ones and a lot more single sentence reviews.

    The one star reviews of course are more in-depth as people love to moan, people love to stress how bad things are, let the world know it so to speak.

    The easiest way to cut down on the 5 star reviews that are misleading (been written by friends or family etc) is to impose a character limit.

    Sure you can leave a review but it must be over 300 characters. Perhaps with a criteria on rating with dropdowns or sliders giving more of an overview of the plugin. What will happen then?

    Reviews will slow down, you will see less that’s for sure.

    What you will be left with however for those who do like to leave constructive feedback, is genuine honest opinions.

    I hardly ever bother reading others feedback on plugins on WordPress.org, for the simple reason half of them are junk.

    I make my own mind up and write a review on my website.

    Just my own thoughts.

    Report


    1. I hate writing the whole encyclopedia when I do reviews. If I want to write 3 words then I will write 3 words.

      If I want to write 30,000 words, then I will write 30,000 words.

      I refuse to write reviews on plugins/themes/services/etc…on friends, family, my clients, my employer, people that have provided me a sample, or paid sponsorship at an event I am volunteering/working at.

      Report

Comments are closed.