38 Comments

  1. Louis Reingold

    SO HAPPY ABOUT THIS!

    Thanks Otto and anyone else who had a part in making this happen.

    The new system is fairer.

    Back in the day, it was possible to leave a rating without a review, but this was changed recently.

    People are more likely to leave a 1 star rating when they don’t have to leave a comment/review with it explaining why.

    So once the requirement to include a review with the rating was added in, plugins got less 1 star reviews. But since the old 1 star ratings were never deleted, old plugins were at a disadvantage and had lower average ratings than newer plugins, even if they were of the same quality.

    This change made our average rating for WP All Import jump from 4.1 to 4.4: https://wordpress.org/support/view/plugin-reviews/wp-all-import

    It also reduced our number of 1 star ratings from 27 to 15. And our number of 5 star ratings went from 90 to 87 – which proves people are way more likely to leave a 1 star rating without a review.

    Report

  2. Ryan Hellyer

    I wrote a comment, but it got lost.

    Short version: Thanks Otto and co :)

    Report

  3. David Anderson

    Good work, Otto!

    I think this ‘reset’ has been good, but I’d love to see the rating shown always be a rolling “last two years” score.

    Some plugins have >5 year histories, and ratings older than ~2 years are basically meaningless. It also makes it hard for plugins like wp-ecommerce to recover from historically low ratings – a rating 5 years ago has exactly the same weighting as one made yesterday.

    Or, for example, some big company buys out a plugin, and takes it in a new direction – they’ve also bought the ratings history, which is nice for them (presuming they bought a good one), but over time is increasingly misleading for would-be new users.

    Report

  4. Der Küchenmeister

    Hurray :(

    Now instead of a solid 4 stars average over the lifetime of the plugin, Media Credit now has a single star because there is only one review, made by someone who was disgruntled about a support request being missed, but who never bothered to reply to his issue again :(

    Is there any kind of appeals process? This kinda sucks.

    https://wordpress.org/support/view/plugin-reviews/media-credit?filter=1

    Report

    • Louis Reingold

      Add a link to inside your plugin saying “If you like the plugin, please leave a review on WordPress.org.”

      You have 10,000+ downloads – I’m sure you can get a few five star reviews posted if you just ask people who genuinely like your plugin!

      Report

    • Mika E (Ipstenu)

      Yes but not for this. You missed a notification (you can use RSS or get emails) so these things can happen. But what you do next can change your review. Ask them about the problem, see if you can solve it, and then if you do, ask the, to change their review from 1 to 4.

      Abusive reviews, unfair reviews, attacks? Yes those are deleted. This one is what we would deem fair. They have a problem, they opened a support request, no one got to them in a couple weeks, they can’t use the plugin.

      To get alerts, go to https://wordpress.org/support/plugin/media-credit and at the bottom there’s a link to the RSS feed or to get email notifications of new support posts.

      Report

    • Mika E (Ipstenu)

      Awesome… Comment was deleted (Jeff – I hit submit and it vanished, no warning no moderation notice, just gone).

      Yes there are ways to report abusive or spam posts however this is not something you would want to flag for ‘review’ or anything because it’s 100% valid.

      Go to https://wordpress.org/support/plugin/media-credit

      At the bottom there’s this: https://cloudup.com/cm2qO243MYO

      You can get email alerts that way. I’ll have to make a post on make/plugins about how you do this, since a lot of people seem to have missed it. But you can get notifications, and if someone submits a support ticket and you missed it, then yes, they should be annoyed.

      How you correct the issue and get a higher rating is all on you, but I would suggest instead of saying ‘we don’t get notifications’ you apologize for missing theirs, link back to it and offer to help. If you resolve the issue, you can then ask them to edit their star rating :) Boom. 4 stars. (IMO I’d personally hold off on 5, since you DID miss the initial support post)

      Report

      • pepe

        Thanks Mika, I have since found that hidden setting, but I didn’t know about it back then. It sure could benefit from some additional publicity for new plugin contributors.

        Back in early 2014, I had just started as a contributor and de facto maintainer of Media Credit after a long period of dormancy for the plugin. I had no idea that there was a way to get hold of all new support requests and so I “patrolled” the forum every other day, initially. Somehow, I still missed that guy’s post. Two and a half weeks later, instead of bump of his original topic to get my/our attention, he posts a new topic, asking whether the plugin was still supported and simultaneously gave the one-star review. I have to admit, that pissed me off somewhat, because we never got the chance to fix the initial oversight. Furthermore, while the bug he reported was fixed about a month later with the next release, he never even bothered to acknowledge this. (I’m not talking about rescinding the bad review here, but about an answer to the initial support post. A “yes, that fixes it” or “no, I’ve still got the following issue” would have been enough.)

        Sorry for the rant, but lack of communication/feedback like this bothers me in an an open source project. (Everything’s different when we are talking about paying customers, of course.)

        Report

  5. Giorgos Sarigiannidis

    It’s a nice addition, but I agree with David Anderson that only the recent ratings (e.g. last 1 or 2 years) should be promoted. For example, if a plugin worked great at the beginning but at some point introduces spammy behavior or has a major security flaw, the overall rating might not correspond to it’s recent state. On the other hand, a plugin that fixed it’s bugs would be unfair to have a negative rating because of flaws it used to have a few years ago. For historical reasons the overall rating could also be visible, but in a less prominent way.

    Another thing that I find misleading has to do with the plugin’s downloads. I believe that plugin updates shouldn’t add up to the downloads count. Instead, a download should count only the initial plugin installation at a specific domain. That’s because at the moment a developer could influence this number by releasing frequent meaningless updates, making the plugin appear more popular than it really is.

    Report

    • Julie @Niackery

      Another thing that I find misleading has to do with the plugin’s downloads. I believe that plugin updates shouldn’t add up to the downloads count. Instead, a download should count only the initial plugin installation at a specific domain.

      I’m not even sure all downloads should count. I sometimes download a plugin to see how it works, check out the code, etc. That doesn’t mean I end up using it. It would be far more interesting and useful to see an “in use” count rather than a downloads count…

      Report

      • Giorgos Sarigiannidis

        @Julie @Niackery
        I agree. Sometimes you download something for testing purposes (e.g. on localhost) or you download it and then you deactive or even remove it. As you suggest, the ideal functionality would be something like “this plugin is active in xxx websites right now”, but I don’t know if it’s possible.

        Report

      • Laura

        I download far more plugins than I actually end up using and I’m not a web developer at all. I don’t look at downloads but I did notice a stat about how often the plugin is actually being used. That is something real versus test runs.

        Report

    • Otto

      The download count is a crappy number, we know. Better statistical data regarding plugin installations is coming soon, once we’re confident in the accuracy of the numbers over time.

      Report

  6. Li-An

    Too bad there is no solution to change bad rating from people who don’t come back after having an answer to their question or did not make any compatibility test with their theme or other plugins.

    Report

  7. Vova Feldman

    Appreciate the upgrade! It’s increased our average rate from 4.4 to 4.5 which is huge! (https://wordpress.org/plugins/rating-widget/)

    I think it’s a smart move, one of my concerns with the previous ratings was that I couldn’t know who are the people behind the votes.

    Having said that, I still think there’s a big issue with the reviews system, many people base their review on expectations, even though it’s not what the plugin does. E.g “That’s the worst plugin ever, it doesn’t support X”. Even if the description page explicitly mention “We do NOT support X”.

    You can’t buy a scooter, and then be pissed off that it’s not a Lamborghini.

    I think that simple bullets with good vs bad reviews examples right before the review form can drastically change that.

    Hope that someone is hearing me there :-)

    Report

    • Laura

      I think the real problem is descriptions and screenshots given for the plugin. Often descriptions leave me guessing as to what the plugin will actually be able to do and how I will be able to use it, or not.

      I tend to skip reading more than the header description and then look for screenshots to see if I can understand the plugin and whether or not it looks too complicated and messy to consider trying at all.

      Often the screenshot says a lot more than the written description. But, if the screenshots interest me I will then go back and read more.

      Some descriptions seem to claim the plugin does more and works better than it actually does. I’ve seen some descriptions actually written for a premium version but posted to the WP plugin directory version. This is confusing and of course it sets higher exceptions than the plugin actually delivers.

      Report

  8. Keith Davis

    The two things that I look at when choosing a plugin are: the rating and the number of outstanding support requests.

    The rating gives a good idea of how people have found the plugin and the support queries answered shows how enthusiastic the author is about providing support.

    I’ll head over and check out the new system

    Report

    • Peter Cralen

      I agree that how developer deal with support question tells a lot …
      For me support part is where I look, reviews mostly say nothing.
      In support I see if developer is active there, what kind of problems user have with plugin …

      Report

  9. Miroslav Glavić

    I look at 3 things when choosing a plugin

    1) Rating

    Higher stars tend to mean it is a good coded plugin

    2) If it’s compatible with my WordPress version

    If it hasn’t been updated in a while then the chances of not working with my WP ver. are greater chances

    3) The amount it has been downloaded

    The more people download the plugin, the greater chances it is a good plugin, people won’t download crappy plugins, right?

    I am sure if enough people complain about a crappy plugin, the plugin gets removed, right?

    Report

  10. Giorgos Sarigiannidis

    That would be an interesting poll (the criteria based on which someone chooses a plugin). Mine are:
    1. Ratings per Rating number (one or two 5-star ratings mean nothing, especially if they come with no actual review).
    2. Downloads (which, as we already mentioned, could be more accurate)
    3. Author’s responsiveness. This could be measured by the update frequency and how many recent support tickets remain unanswered.

    In my opinion, another problem with the plugin directory is that it’s difficult to distinguish quality plugins from the bad ones. Here’s some of my thoughts about features that could improve that:
    1. Restrict the number of tags a plugin can have, in order to avoid tag-spamming.
    2. Introduce hierarchical categories – e.g. Performance, SEO, Appearance etc. where a plugin can only be under a single category.
    3. Better featured plugins. Right now, featured plugins section shows the same plugins. It would be more useful if it promoted different plugins every day, based on their quality. The criteria could combine things such as the number of positive reviews during the past week, handpick based on some special occasion (e.g. a good security plugin in case of a virus outbreak) or even the author’s reputation (I would be willing to try a new plugin by Otto or Nacin even if it had zero reviews and downloads at the moment).

    Report

  11. Justin Tadlock

    It’s great to see my Members plugin ratings jump up a bit. I believe it had 19 one-star reviews from anonymous users. Now, it only has 2 one-star reviews. It’s awesome to see who’s behind those bad reviews too. One is from someone who rated the plugin down because it didn’t have the feature he wanted. The second was from a user who made a fixable mistake and blamed the plugin (granted, it’s a feature I could improve). Anyway, now I have that 4.8 rating with new change.

    Everyone loves those 5-star reviews, but I think it’s the 4-star reviews that are the most helpful to developers. Those are usually from people who genuinely love the plugin but have ideas about making it better.

    Report

  12. Cais

    Although its really great every star-rating now has a review matching it, I think its a bit unfair that the older reviews are also closed but apparently count towards the over-all star-rating for the plugin.

    I would prefer to see the reviews remain open so they can be addressed as needed … or to see the star-rating reflect the “current” open reviews only, much like the rolling (current) support topics count.

    Report

    • Samuel "Otto" Wood

      Review threads, like all other threads in the forum, auto-close themselves after 1 year. Honestly, a year seems like plenty of time to respond.

      Now, don’t get me wrong. Reviews, including their ratings, can still be edited by their original authors regardless of the closed state. So, if you address the problem, then there’s still a chance the person can come back and adjust their review. It’s possible.

      Report

      • Ryan Hellyer

        I did that recently. I wrote a 1 star review, then waited a few years to make sure they didn’t make any more botch ups (they broke an upgrade in spectacular fashion) and then bumped it up to a 4 star review. I’d like to have left a comment pointing this out though, but didn’t have the opportunity since the comments were closed by then.

        Report

      • Cais

        I agree a year should be plenty of time to respond … just thinking more consistency as compared to other “metrics” being displayed on the plugin’s description page as being more “fair”.

        Report

  13. Chook

    Are Ratings to be used for (forceful) Help? I’ve seen a few 1 star reviews on nice plugins only because the user didn’t know (or bother) how to use it. :/ Something needs to be done about that.

    Would a voting system on a given review help reviewers give a more helpful rating?
    Such as, other users can vote on a review. “Found this review Helpful?” Y/N.

    Report

    • Julie @Niackery

      I like the idea of voting reviews up or down, but the system would have to be sophisticated enough to weed out older upvoted reviews. I can just imagine a situation where a plugin introduces an update that breaks things, gets a bunch of bad reviews which are upvoted, then fixes the problem, but the bad reviews remain prominent… :(

      Report

      • Chook

        Hmm, I see your point… In such a situation the Plugin Author would need to work quickly to fix such an oversight. Then the Fixed Updated Plugin reviews (hopefully positive because the Author fixed it quickly) help ‘naturalize’ that effect?

        Report

  14. Giorgos Sarigiannidis

    I don’t know if it’s old news but I just noticed that instead of the downloads number you now see “active installs” in a plugin’s page.

    Report

  15. David Cameron Law

    Just noticed the change, I’m now in a happy place as pretty sure had other WordPress SEO Plugin authors/users downgrading my little plugin (Stallion WordPress SEO Plugin) and was rated 1 star on something like 40 ratings with no reviews (lots of 1 stars with no review) which made no sense for a plugin with less than 20K downloads!

    Now it’s rated 4.8 from 5 reviews which makes far more sense for a plugin not many people use.

    David

    Report

  16. Laura

    Plugin ratings and reviews are important. Themes go through a review system before they are listed but plugins don’t have the same standards. So it’s more “user beware”. Not every plugin has a review/ rating but it does give you something to go on when you try to decide which plugin to try. Even the best plugins are a risk to upload and activate.

    Report

Comments are closed.

%d bloggers like this: