Recently we got slammed on wordpress.org with a couple of back-to-back one-star reviews.  Which got me thinking about the purpose of them.  The official line from the WordPress.org powers-that-be is that reviews are supposed to be helpful to developers.

Yet, the majority of one-star and two-star reviews don't provide any kind of useful feedback.

In fact, in most cases a one or two star review can only be described as being punitive in nature - designed to punish, not help.

And we can't think of a single instance where feedback from a one star review actually helped us improve our products.  Instead, every time we see a new one it causes us to question whether we really need to have our product listed on wordpress.org or if we should even offer any kind of free version in the first place.

As a review tool, they're not working in providing actionable feedback.  Though, as a punishment tool it can be quite effective in the angst it causes, the effect of lost sales and the overall distraction created in responding to them.

About the only time they're effective as a review tool is if there are a large number of one and two star reviews relative to the overall set.  Then an end user can see a clear pattern.

But if the pattern is what a potential new user needs to see then maybe one and two star reviews should be allowed but without commentary.  The content of these kinds of reviews aren't really useful for building a better product and in most cases are inflammatory, angry and riddled with emotions. If a user was really interested in helping, a standard forum thread would do that part of the job just as effectively.

Right now though, the one and two star reviews do not seem to be accomplishing the goal that the organizers behind them are trying to accomplish.

Reasons For One and Two Star Reviews

In our experience there are six reasons for one and two star reviews:

• Users are annoyed because the product doesn't work immediately after activation.  This might mean that they need to read the directions. Or maybe there's a hidden conflict.  Either way, they immediately slap a one star review, move on and don't even attempt to ask for help.  This kind of review is definitely not useful.
• Similar to (1) above, except the user asks for help but is annoyed by the trouble shooting process and gives up.  Same result, same lack of usefulness.  To be fair, when doing remote trouble-shooting, patience is required by both parties, especially with WordPress and its penchant for plugin conflicts.
• Users are annoyed because features they expect in the free product are only available in the paid product.  So they slap a one star review because they actually have to pay for something.  This might be slightly useful if others had the same opinion and provided similar one-star reviews.  Maybe a premium feature should be in the free product - but so far we have never had someone confirm this kind of a review with a similar one star review of their own.  Useful?  Maybe - in the sense that it causes us to pause and question whether something should be made free.
• Fraudulent reviews - the kind orchestrated by competitors to drag down a product's ratings.  These are definitely not useful for us.
• Users expecting to use the product in ways that aren't documented.  Our most recent one-star review falls into this category.
• Trolls - folks whose only intent is to get a rise out of you.  Our most recent one-star review arguably falls into this category as well. Unfortunately, we took the bait and ended up in a long winded unproductive argument.

I would be hard-pressed to find a developer who can honestly say that one and two star reviews helped to make their product better or helped to create a better business.  But we can find a lot of those that decided to exit the business or abandon maintaining plugins because they can't handle customers who are brutal in their feedback.  And why should they if they're offering something for free?

Alternatives

There are two articles we wish users would read before leaving reviews but that's probably wishful thinking.

• There’s a wrong way to give plugin feedback on WordPress.org
• How to Leave a Good Bad Review

However, maybe users leaving one and two star reviews might be directed to these articles instead of being allowed to add a comment to their rating.

Another alternative would be to allow the rating and the commentary but not have it shown or allowed to affect the overall product score - until the total number of one and two star ratings exceed a certain level (say, 15% or 20% of the reviews?)

Yet another alternative would be to allow one and two star ratings only from users that have been active contributors in the past.  That would be a proxy for credibility.

Or maybe some combination of all those ideas - for example:

• When attempting to leave a one or two star review, ask users to read the guidelines for leaving such reviews before allowing them to leave their comments
• Immediately let one and two star ratings from credible users affect a plugin's overall score
• Delay one and two star ratings from new users (or users who don't usually rate plugins) from affecting a plugin's score until the number of one and two star ratings exceed a certain threshold (say, 15% or even 20%).  In this case, the author would see all reviews but they would be hidden from the public until later.
• Potentially age out one and two star reviews after, say, 3 years?

Wrap Up

One and two star ratings are not effective in providing actionable feedback and, instead, is often used as a tool for punishing plugin authors for some perceived misdeed.  It would be better if one and two star ratings were "gated" somehow to prevent the punishment effect while still being useful as a feedback mechanism.  A few ideas on how this can work were provided above.

If you're a plugin author, we would love to hear your feedback on this - just drop us a note via our contact form!