Product Reviews in the WordPress Ecosystem: Honesty and Genuine Experiences

I don’t write fluff pieces. I call ’em like I see ’em. If your project is a dumpster fire, I’m going to say it’s a dumpster fire.

Whenever someone comes to me in hopes that I review their product, I give them some form of the preceding paragraph. It doesn’t matter if it is a plugin, theme, web host, or some other product. What matters is that I write my review with honesty and offer my genuine opinion about the thing they built.

I rarely read most product reviews in the WordPress community anymore. Far too often the reviewers are not offering their genuine experience with a product. You get something like “XYZ is a multi-purpose theme that is built for…yadda, yadda, yadda…” Yawn. It reads as if publishers are trying to sell a product. If you dig deep enough, you realize that is what many are doing (hello, affiliate links).

I also have it on good authority (I’ve seen some of the email exchanges) that a lot of money exchanges hands behinds the scenes for reviews. Most of the time, publishers are not writing a review of the product. They are selling you their dreams of a continued working relationship with the product maker.

There’s nothing wrong with affiliate links if a publisher loves a product. There’s no issue with paid reviews if such reviews are honest experiences with the product. There’s also no problem with writing a love letter to your favorite plugin and theme with no financial incentive.

However, what I generally see are shallow reviews at best. Many, dare I say most, reviews are not genuine. They are certainly not real journalism.

The best place to find genuine reviews are from the user ratings on WordPress.org, assuming the plugin or theme is available there. Users tend to not hold back, particularly if their review is negative.

It is tough as an artist (yes, I consider all programmers artists). I’ve been on the receiving end of negative reviews of things I’ve built. You learn to grow thick skin after a decade of putting your art out into the world.

When I was younger, I tended to be a bit hot-headed whenever I got a bad review for something I had built. After pouring my heart and soul into a project, it cut deep to read a negative review. I wasn’t always the most gracious receiver of such reviews. There are responses I wish I could take back. Looking at those times now, I wish I would have been more open to hearing what the reviewer was saying. Even if I disagreed with every word, it did not mean that the person wasn’t providing me something of value with their review.

With age and I hope a little more wisdom, I usually give myself time to think about what someone is saying before I respond. I allow my thoughts time to develop and mature. Often, it turns out, critical reviews are far more helpful in making better art than all the five-star ratings in the world.

When I took the writing position at WP Tavern, I wanted to bring a review format to the website that is missing within our community. I wanted to do reviews based on my experience as both a user and a developer. I admit that I was not prepared for a negative reaction to what was in part a negative review. As always, I gave myself time to read and think over what some commenters were saying. This article is my response.

Reviews Are About Personal Experience

One of the things I learned early on as a writer is to not second guess myself, especially when writing an opinion piece. It is not good for one’s mental health.

An opinion piece is about the moment. It is raw. It is passionate.

Writers’ opinions may change over time. They are human and have the freedom to change their minds later. However, an opinion-based story should reflect that single moment in time and what the author’s feelings are at that moment.

There’s a common (and wholly incorrect) notion that journalism should be nothing more than facts, that subjectivity is not allowed. Throughout the several hundred years that some form of journalism has existed, there has never existed a point where the whole of the field was objective. Even in the early days of U.S. journalism, my country’s founders published articles in newspapers to sway public opinion on ratifying the U.S. Constitution.

Reporting, which is one form of journalism, does not represent the whole. It is the most objective form of journalism in which the reporter simply tells the news to readers. We certainly do plenty of that at the Tavern. However, other forms like editorials, features, and reviews are as important. These forms take a different approach.

Reviews are the unwieldy beasts of journalism. They are hard to tame. They’re not always pretty. However, they should always be true to their nature. They can bring out angry hoards of fanboys down on the critic (ever read the comments of a critical review of an Apple product?).

Honest reviews are about personal experience. If a film critic dislikes the latest movie in the Marvel Cinematic Universe, it is that critic’s duty to write about their experience watching it. The reviewer has an obligation to not huddle in fear of Iron Man fanatics who will inevitably send ad hominem attacks his way. Holding back one’s opinion within a review is the ultimate sin of a critic.

Like with any products or forms of art, WordPress plugins and themes are not immune to this same criticism. Such criticism is even more important when the software costs money and potential buyers may be looking for genuine reviews.

Disagreement with a review is OK. Disagreements are more interesting than everyone nodding their heads in unison. What a boring world it would be if we were all in agreement.

However, I did want to address comments on my previous review about it being unfair, specifically the unfairness of my personal experience. It’s that personal experience that makes a review genuine. Not everyone’s experience will be the same. One person’s one-star rating does not discount another’s five stars. They are equally valid because they represent different experiences.

Developers Are Users Too

There’s a common idea in the WordPress community that developers are not users, that our experiences don’t count because our knowledge and skillsets are more advanced than the average. At first glance, the argument makes some sense. However, after giving it some serious thought, I reject the notion.

Martin Scorsese can’t criticize films because he makes films. There’s no way he can feel what the average person does at the cinema.

Beyoncé can’t judge a music competition because she’s a singer. She’s not listening with the ears of a normal human.

Wait; that’s not right, is it?

Why is it that developers’ opinions are so easily discounted when they are critical of user experience? I use WordPress, different themes, and various plugins every day. I use those that make me happy or serve essential functions. I don’t necessarily pick plugins out because I like their code. I use them because I too am a user in every way that a non-developer is a user. Having the ability to articulate the problems from a different viewpoint doesn’t change that.

In many ways, developers can provide more useful software reviews than “average” users because we have some past experience solving the same problems.

Offering a Genuine Review

One thing you will always get from me is honesty. When I review a WordPress-related product, you will always read about my personal experience.

I was fully prepared to say that the gloves are coming off, but the gloves have always been off. I will never hold back criticism. I’m always ready to pile on the praise too.

But, I won’t lie to you.

Who’s ready to have their theme or plugin reviewed next?

13

13 responses to “Product Reviews in the WordPress Ecosystem: Honesty and Genuine Experiences”

  1. I did read that review and found that I appreciated learning about the plugin from a developer’s perspective. Having experience makes the review far more impactful. But my personal bias, which gives your point of view more credibility, lies in the fact that I’ve known about you for quite some time as a developer in the WordPress community, I have used some of your plugins, and I’ve read your support answers to others questions. We’ve also communicated personally. So, I’m going to weigh your experience favorably when I read a review because I understand the depth of your experience and would probably see it in the same way you do.

    Our shared work history is problem-solving and when we see software that introduces more problems than they solve, we are going to be quite honest about them. Some people may not agree because they don’t look at the user experience the same way we do. It’s our job to criticize the user experience not for the sake of criticism only, but to make what we’re working with better. And, we can do that far more easily with software, which can be fixed more quickly than with hardware, which has a cost and intensity in physical materials we don’t have.

    However, if someone didn’t know you and they didn’t understand your skill level in this universe, they might read your review as coming from a snobbish developer. That developers always have something bad to say about someone else’s work, because truth-be-told, developers are “artists” and artists have proven to be quite snobbish from time to time about the way other artists work.

    But, that’s the risk you take going from developer to journalist. At the end of the day, someone is always going to criticize you. There is always going to be second-guessing and someone trying to invalidate your point of view. It happens to me every day.

    They are going to hit you far harder now that you’re a journalist than when you were just a developer because you have a far larger audience now for your words than for your theme and plugins, which were targeting a subset of the community and had a finite audience. You’re now on a much larger stage where it’s not just about you and your software, it’s about you picking apart other people’s software – and for good reason, nonetheless.

    But, it goes without saying that whatever path you take, someone is going to be there to offer a criticism. Since you’ve got an already thick skin, it’s just grown a tab bit thicker after this post, LOL. You’re just getting your feet wet and you’ll learn a style that works for you that may reduce the blowback.

    I’ve been reading this great book, “Writing to Persuade: How to Bring People Over to Your Side” by Trish Hall, a former NYTimes OpEd Editor. (https://www.goodreads.com/book/show/41817516-writing-to-persuade). I urge you to check it out and it may help you reformat your approach to reviews or your general writing, for that matter. I’ve found it to be extremely helpful to me, as well.

  2. I argued many of the same things for years, before removing our 40+ plugins from WP.org. The directory is infested with spam, fraud, malware, and anonymous trolls. What’s the point?

    One point needs correction, however: there actually is something wrong with affiliate links and paid reviews, because in many cases they are illegally published. Anytime someone is publicly endorsing a product without disclosing the quid-pro-quo relationship behind their words, it is a form of consumer fraud and is specifically outlawed in the United States, the European Union, and elsewhere:

    https://www.ftc.gov/tips-advice/business-center/guidance/ftcs-endorsement-guides-what-people-are-asking

    Personally I’ve never had an issue with critical reviews that are sincerely offering feedback. The problem with much of the WordPress community is that so many users are purposefully sabotaging their competitors, paying for fake reviews, or simply trying to trash small businesses who refused to offer them “free support” — in other words, the reviews lack merit to begin with.

    When I mentioned these sentiments to WP.org admins, I was mocked and repeatedly told that “reviews are part of listing your products to customers” when in fact my team was donating free and open source code to the community — not even freemium code. These users most certainly were never our customers, that is for sure…

    Another good reason why anonymity hurts FOSS feedback.

    • It’s a disappointment with any platform, whether it grants anonymity or otherwise. Facebook pages are full of such reviews too.

      WordPress itself is distributed as open source under GPL, and if you go on forums discussing CMS and web platforms, there are similar reviews for WordPress (just try asking in webdev subreddit). So, I kinda understand both WP.org mods and plugin/theme developers.

      I guess we all need to be more understanding of each other, rather than mocking one another. That’s not good.

  3. Hello Justin,

    I wholeheartedly agree that reviews are about personal experiences and that a reviewer shouldn’t himself or herself back simply because of the affiliate relationship.

    Product and service reviews in the WordPress ecosystem are crying for originality and honesty. Most of the reviews are by people who haven’t tested the product or service they are reviewing.

    And many of the reviews are merely a rehash of other reviews. I agree that currently, WordPress.org reviews are the most honest ones that you can read.

  4. Yes, reviews should be in the best sense personal reviews of the reviewer for the product. Therefore they have to be subjective and emotional. A completely objective, purely objective review is not possible, is it? It would be more like a data sheet, wouldn’t it?

    It is also good and sensible for journalists to review things, of course. But it always becomes problematic when such things are not marked as reviews, opinions or comments. But when one pretends to report something and then delivers an opinion article. Therefore, clearly mark what it is, that should be standard anyway.
    The different journalistic forms should be clearly named and distinguished from each other: Report, commentary, review, glossary, essay, etc.

    Regarding plugin and theme reviews on WordPress.org:
    There will certainly be some abuse, and somewhere and somehow a grey area. You can never prevent that 100%. However, the .org team does a lot to prevent that. And they really do a lot.
    I also agree with a lot of what the .org team says on the subject: reviews are a kind of feedback, yes. And when I go public with my software “product” as a plugin author, I automatically expose myself to some kind of rating. That is correct. The only question is, what is “allowed” to endure? – Are false claims (= lies), insults and similar things really part of it? No, they don’t belong on there. Here also more stuff would have to be tidied up. Not in every case the user/customer is right. I have already read far too many reviews on .org in the last years, where you could and should have intervened and where exactly that didn’t happen. Unfortunately, the tendency is more and more to allow users to let off steam and be encouraged to do so, whereas the plugin authors are often left behind. That is unfair and unjust! In the last few months I have read in about every second statement that plugin authors will be removed if they don’t want to be offered all that. This is called blackmail. Point.

    Which unfortunately is still a huge problem that users use the 1-star reviews to “punish” the plugin author for whatever. Users take the view that it’s their only or most effective way to put “pressure” on the plugin author to do anything with it (even if it’s just to indulge or troll their own ego). At first glance, I can even understand that. But if you think longer about it, it’s just disrespectful and insolent and doesn’t belong there.

    What these users usually don’t know and also don’t want to admit that reviews are a very important ranking factor for the search results on WordPress.org, as well as for the listings, e.g. “Popular Plugins”. I.e. the “punishment action” of an unsatisfied user can cause quite some problems for a hobby plugin author. Most of the time even permanent problems, because his ranking go down and his reputation suffers. And it can affect anyone, even providers who offer the very best free support you can imagine and already have a super solid plugin product. Nobody is immune from such “punishment or envy actions”.

    There are not only companies, agencies etc., which release plugins on .org, but also still many thousands hobbyist developers. For these authors, 1-star reviews are total poison, and usually destroy a lot of motivation. Here simply more consideration and mutual understanding and respect is missing.

    • If there is in appropriate content in a review, it can be removed by the moderating team.

      Also, a one-star review can be changed by the user setting the rating. The plugin author may not get someone do that in every case, but it’s worth mentioning that you can start least ask

  5. honestly im happy i work as i do on websites i try for myself i dont read reviews yes i check them in test environments to be sure they are not doing things wrong etc i also dont use wordpress.org besides the plugin installer in wordpress dashboard

    but i never knew it was this bad for programmers their that’s bad i feel for you guys on that i just develop websites with paid and free plugins and i always read news and stuff on this topic but i could imagine why i have some plugins that just are not even on wordpress.org after reading this

  6. Maybe I’m jaded because I work for an SEO company called AdInfusion but hear me out..

    I have been seeing such an increase lately in the misuse and overuse of review schema in the SERPS that it’s only a matter of time before this all gets nuked. Trust is gonna get a lot harder to earn…

    what did Gary V. say ? Marketers ruin everything..

    • Remember that review scheme in a serp is only tied to a secondary page and not the home page of the website. You will never see star ratings on a home page. You will only see star ratings on a langing page. So, while anyone can download the WP Reviews plugin and assign start ratings and fake reviews to their site, the stars will only be visible on a landing page and not the home page. Landing pages are usually marketed via pay per click. I don’t find a lot of landing pages with star ratings in a Google search. I think they must know that while star ratings are cool, they are not a ranking signal.

    • What ruins everything is not the marketers, it’s the business’s pursuit of bigger, faster and easier profits. The marketer is only the promoter and no more the villain than the software developer.

      If only a few more businesses would focus on earning their income by getter better, rather than earning more by getting bigger, we wouldn’t have this problem.

      And reviews could again be just that, instead of the meaningless attempts to grab the unsuspecting user’s money they are today.

  7. Just a few weeks ago Google updated their search to fight back against self-serving reviews.

    Now many sites that were gaming the review schemas aren’t having their stars show up in the search results.

    Only the following 17 categories can have the stars show up:

    schema.org/Book
    schema.org/Course
    schema.org/CreativeWorkSeason
    schema.org/CreativeWorkSeries
    schema.org/Episode
    schema.org/Event
    schema.org/Game
    schema.org/HowTo
    schema.org/LocalBusiness
    schema.org/MediaObject
    schema.org/Movie
    schema.org/MusicPlaylist
    schema.org/MusicRecording
    schema.org/Organization
    schema.org/Product
    schema.org/Recipe
    schema.org/SoftwareApplication

  8. I think this is going to have an even bigger impact for search with the BERT update that just happened.

    Google is looking to things like schema to better understand content and structured data is the way to do it. I have a few of my clients (I run an agency called Octiv Digital) showing featured snippets for real estate structured data and I’m excited to see what happens with this new alg update. Just wait for SEOs to spam it though…

Newsletter

Subscribe Via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.