Do review scores matter? Does all the1-through-10fuss mean anything when the games hit retail? Absolutely, says an EEDAR/SMU study which concludes "a strong causal link between professional critic reviews and their ability to affect consumers’ actions and perceptions."
188 students who had never played Plants vs. Zombies were asked to read an information packet before playing the game for 20 minutes. One group received packets containing positive reviews (a score of 90), another received negative reviews (a score of 61), and a control group received no reviews.
After their play session, the participants were asked to rate the game. The group exposed to high review scores rated the game, on average, six points higher than the control group, and 14 points higher than the group exposed to low scores. Members of the positive-review-influenced group were also more than twice as likely than the low score group to choose a copy of the game over $10 cash, and were more likely to recommend the game to a friend (91% would recommend, compared to 61% of the low review score group).
Above: The blue bars represent the review scores given to participants, and the red bars are their scores after 20 minutes of playing
The recommendation factor is important, because most consumers indicate that they trust recommendations from friends more than review scores. This study, however, links review scores and a person's willingness to recommend a game:
"Word-of-mouth is usually considered a more influential catalyst than critic reviews. The EEDAR/SMU study indicates that the amount of word-of-mouth spread is directly impacted by critic reviews, making reviews an important element in the purchasing decision process."
Above: Willingness to recommend Plants vs. Zombies (blue) compared to unwillingness (red). From left to right, the group exposed to high scores, the control group, and the group exposed to low scores
The study did have limitations. The reviews given to participants contained both numbered scores and qualitative statements, such as "Game of the Year." It is unclear which was more impactful, if either. The study also used a game generally accepted as "high quality" (well, we think so), and while the researchers assume that the same results may also apply to lower quality titles, another study would be needed to confirm that. The study also did not examine the effects of marketing (trailers, screenshots, give-aways, etc.), and only indicates that critic reviews influence those directly exposed - it does not make conclusions about word-of-mouth recommendations which occur down the line.
Above: If anything can be concluded, it's that this works
A link between review scores and game sales was already assumed, and tested, but these results further reinforce the importance of a positive critical reaction. Garth Chouteau, Vice President of Public Relations at PopCap Games, explained what the study means to PopCap:
"We've always known that good reviews are beneficial to a game's sales, but we didn't realize just how significant a role they play in the purchasing decision process. This study illustrates not only that game quality trumps hype, it also shows how important it is for game makers to establish and maintain good communications with the media outlets that cover video games."
That was a lot of numbers and stuff, so now we'll throw it to you for gut reactions, which'll be much more fun. What's your take?How much influence do reviews have on your decisions? Do you read many reviews, just a few, or do you prefer recommendations from friends? Do you consider that your friends' recommendationsmay have been influenced by reviews? Tell us in the comments!
Jul 7, 2010
