Which rating mechanism do users find more valuable in a scenario where the user is looking for an article in a Knowledge-Base where there are multiple articles that cover a single topic?
Is a 5-star rating more user-friendly and valuable than a simple up/down vote model?
Answer
I beg to disagree with the other answers, so, I would argue the main differentiating factor is the time it costs for the user to rate. Voting up or down can be done in seconds and is especially useful if there is a clear concept of what up and what down means. This does not need to be the same, for example on Stackoverflow this is "correct answer" or "bad answer", whereas in comment sections it tends to mean "I like" and "I dislike". In general this means that one should use a star rating system if either scoring items is rewarding to the user (e.g. better recommendations) and thus worth his time, or if rating items is a fairly rare occurrence. On the other hand, if you expect users to rate countless of items in general it will be hard to convince them to use a multi tier ranking system and two tier or even one tier (Facebook "like" button) systems will work better.
If you don't differentiate the different levels clearly enough it will quickly happen users only vote 0, n-1 or n stars. This still allows you to run a lot of statistics on it (e.g. when accounting for user average weighting etc.), but it's nowhere near as good as when users distribute their votes well. For this reason implementing a multi tier system is far harder than up/down vote option. One can improve on this by adding labels to the rating system or making explicit rules on how the voting should work (see netflix.com(Deprecated, they used to show descriptions whilst hovering over stars) or tweakers.net(Dutch)) for two good examples of this. On the other hand with a two tier system there tends to be simply no right or wrong, it's all up to the user. Even on SE you can upvote only bad answers and downvote only good answers and you likely won't get banned (correct me if I am wrong, but I believe there are no wrong votes on SE).
Another issue raised in some of the articles is that with up/down the result is often displayed as a score, whereas with stars it's often shown as an average. Now, internally you will probably be using proper weighting in all cases when calculating averages, but the real crux to the problem is presentation towards the user.
The aforementioned dutch website tweakers.net for example assigns scores -1 to +3 to their rating system (-1 being flame, 0 being offtopic, +1 ontopic and +2/+3 an excellent/exemplary read) and use averages, specifically the median, to decide how to present objects (objects with a median of 2 or above will get a darker background). +3's are so rare that they do not need to worry about showing the number of users that voted etc. On the other hands, if you take a site like netflix they simply won't show average star rating till they have got enough data and even then they try to switch as quickly as possible to a personal "predicted rating" rather than showing averages. And the last two more normal options are showing the number of users voted (really common) and total scores (the sum of all star ratings, less common, but I have seen it a couple of times now where rather than stars you have something like "+3615 cookies").
As the OP is asking about a specific case let's examine it. If we're talking about a relatively dedicated group of users then they are likely to be willing to do star ratings. However! I would advice you to present this as a crowd moderation tool, rather than as a measurement of how much users like it. In other words, make the top level say "Excellent and Recommended article!" an average score "Nice read, learned more than expected", two 'stars' something like "Found my answer" and 1 'star' would say "Needs work." (of course those are just some sample labels). On the other hand, if you've a huge number of users that just hop in quickly, want their answer and leave just as quickly you're more likely to get data using just a two tier system.
No comments:
Post a Comment