Dirk Knemeyer

Five star rating system

Five star rating system, April 28, 2010

A short comment contextualizing the prediction could go right here

In my own (extensive) use of Yelp, I’ve come to realize that while the input engine might be 5 stars, the output engine is muddled. There are no 5 star results; there are no 1 star results. Instead the results break down to:

4 1/2 Stars – Highest rated; going to be great
4 Stars – Definitely good, buy with confidence
3 1/2 stars – Could be good or could be bad, buyer beware
3 stars or less – Dreadful, avoid at all costs
Now, it took me a while to figure this out, that [problem] the 5 star system is not 5 stars at all. In fact it is a range of just 2 stars that covers the entirety of practical results!

I think the solution is rather simple. We need to look at what the data is giving us – four distinct and specific choices – and re-fashion the system around that. It could be done one of two ways:

1. Fix the output system only. Convert it into the four things I identified above: Great, Good, OK/Mixed, Bad. Simple, straightforward. So, users pick their 1-5 stars when they rate the restaurant, but that translates into a more plain-language system that provides directive behaviour. The average star rating could still show up in a supplementary capacity to provide transparency into the system. [b: Buzzfeed does this, sorta]

2. Revamp the entire system around those four things. So, a user inputs their rating as one of Great, Good, OK, Bad and the system outputs the same way. I’m not schooled enough on the micro data patterns behind the roll-up to suggest if this is a better approach or not.

In either case, the intent is to give the user more useful and directive information.