psychology - Are stars a good rating system?

Quoting myself from this other question, there are users that rate us with 1 star when the review is "Good product", as well as the other way around, rating us 5 stars for "Bad product" reviews, sometimes.

It's incredibly strange from my point of view, since I've always seen the star system the way it's intended (the more filled stars, the better), but it's quite a trend that some users don't see it the same way.

I know that for numeric scales it may happen, somehow, in Germany (though I've seen both cases, where 1 means "the best" or "the worst"), but I didn't know it could happen in this case.

So, after this relatively personal explanation, could it be that stars are not a good rating system?

Or is it just that they are not taking it that seriously?

Answer

I wouldn't say so because it is simply too confusing. As everyone else has posted, star ratings are subjective and some may see 3/5 being worse that others.

When you want users rating on something, it is best to simplify it even further so the user can intuitively know whats going on.

That is why Youtube switched over from stars to liking/disliking a video which had backlash in the beginning but it gradually improved over time and it is a better representation of the video by the community. You see this with many other popular sites as well.

See the following from Margaret Gould Stewart, UX Lead at YouTube (previously Facebook and Google):

"Now, years ago, when I was working at YouTube, we were looking for ways to encourage more people to rate videos, and it was interesting because when we looked into the data, we found that almost everyone was exclusively using the highest five-star rating, a handful of people were using the lowest one-star, and virtually no one was using two, three or four stars. So we decided to simplify into an up-down kind of voting binary model. It's going to be much easier for people to engage with.

But people were very attached to the five-star rating system. Video creators really loved their ratings. Millions and millions of people were accustomed to the old design. So in order to help people prepare themselves for change and acclimate to the new design more quickly, we actually published the data graph sharing with the community the rationale for what we were going to do, and it even engaged the larger industry in a conversation, which resulted in my favorite TechCrunch headline of all time: 'YouTube Comes to a 5-Star Realization: Its Ratings Are Useless.'"

http://www.ted.com/talks/margaret_gould_stewart_how_giant_websites_design_for_you_and_a_billion_others_too/transcript?language=en