Those who host plugins in the WordPress plugin directory may have noticed a change to their plugin’s ratings. That’s because the ratings system has been reset and rebuilt by Samuel “Otto” Wood. The ratings now correspond exactly with reviews. According to Wood, the change has been two and a half years in the making:
Back when we launched the review system 2.5 years ago, we tied ratings to reviews. However, up until that point, we had existing ratings in the system. At the time, some argued that the ratings should be wiped out and everybody start fresh. I argued for the opposite, that we should leave the existing ratings in place until such time as we had enough reviews in the system to build up a good body of ratings.
A few weeks ago, Wood was checking out the ratings for the new WordPress theme directory when he noticed there were over 150K reviews. “Out of those 150K reviews, less than 10K are for themes” Wood said. This was enough to initiate the change and remove ratings not attached to a review. Since the system was rebuilt, some plugins have experienced 1 and 5 star rating swings, while plugins like WP eCommerce have seen its average rating rise from 2.9 to 3.4 stars.
That means @wpecommerce, which had been at a rating of 2.8 for years, and more recently 2.9 – is now 3.4 stars.
That feels good :)
— Justin Sainton (@JS_Zao) February 26, 2015
In addition to displaying more accurate data, review spam has been neutralized. Review spam has been a serious problem for the past several years. According to Otto, this is no longer the case:
Ask any of the moderation team how many duplicate accounts they’ve seen to vote up their plugin and down those of competitors. Even though the reviews were removed, until yesterday, the ratings made by those accounts remained. This is no longer the case, and the ratings made by those spam accounts has now been effectively neutralized.
In summary, ratings and averages now reflect the most accurate data from the point in time reviews were introduced. Spam and fraudulent data hasn’t been lost or deleted but rather ignored from influencing the data set. Ratings that were added before mandatory reviews are also ignored.
By waiting two and a half years, Wood has avoided resetting the entire system which would have caused everyone to start from zero. This way, authors are able to maintain their ratings and reviews without starting from scratch. If you host a plugin on the WordPress plugin directory, let us know if your ratings changed, especially the average.