Wherein I Criticize A Critique of Metacritic
Posted by John on May 09, 2010 (permanent link to this entry)
As Michael Abbott explains, Metacritic has a lot of problems. We all know that, have known that, and will know that. I’m sure the people who created and maintain Metacritic are well aware of its shortcomings. Metacritic has become a discussion topic just for the sake of being a discussion topic, closing in on that area of worthlessness occupied by piracy and art.
On one hand, I understand why Metacritic is so agitating and why it inspires so many harsh reactions. It is a remarkable force in the videogame market and it can be upsetting when a good game gets a low score, which can essentially destine it for commercial failure. On the other hand, I don’t fully understand what all of our anger is directed towards exactly. Metacritic is constantly being cited as a dysfunctional segment of the industry, but is it really to blame for any of the problems that are allegedly its fault? It’s a utility that, like any utility, has its strengths and weaknesses. Whenever I describe why Metacritic sucks, I’m not saying anything that’s not obvious or not true about any review aggregator.
The problem I have with criticizing Metacritic is the problem I have with Metacritic itself. Metacritic is a heuristic approach to judging games. It reduces them to a single integer. The only way it could possibly simplify games more would be if it used a binary “good” or “bad” score. Even if we repair or overlook all of the issues with its methods, we still have something that’s fundamentally flawed in its very nature.
Criticizing Metacritic is similarly heuristic. Metacritic is a perceived problem with the videogame industry and it happens to be an extremely easy target for us complain about. Okay, Metacritic is broken. How do we want to fix it? Suppose the people who run Metacritic go ahead and consider all of Michael’s bulleted complaints and overhaul their system. Would we stop complaining about it then? I doubt it. Ultimately, the problem we have with Metacritic isn’t really a problem with Metacritic. It’s a problem with the entire videogame culture. It’s a problem in the way in which we use Metacritic. Metacritic is what it is; it’s neither good or evil. It can’t be blamed for destroying developers’ jobs. It was never designed nor intended to do that. We did that.
The real issue is much more complicated than what can be blamed on any single entity. Metacritic wouldn’t have a disproportional effect on sales if we didn’t allow places like it to dominate our purchasing decisions. Metacritic wouldn’t have that sort of weight if individual reviews weren’t so bland and unreliable. Individual reviews would be better if gamers were more interested in reading intelligent prose and if good writers weren’t sidelined to unprofessional blogs. Gamers would be more interested in intelligent writers if games themselves were more intelligent. But I suppose there’s not much money in that. And the reasons for that can be traced in all sorts of directions.
That of course is my own gross oversimplification of the problem, if what I’m talking about can even be considered a solitary definable problem.
I personally haven’t used Metacritic for a long time. Ever since I unplugged myself from the perpetual hype cycle of new triple-A releases I just don’t see any appeal to it. Why would I buy a game just because of a number? Games are so complex that there are lots of high scoring games I simply don’t find interesting due to my personal tastes. Michael gives some good advice: “If you're looking for guidance […] identify a few reviewers or critics whose sensibilities seem to align with your own and carefully read their responses to the games they play.” This advice is actually so good that my response is: duh! I don’t see the logic in any other possible way of picking up games. The only way I can even find myself interested in a game is if someone recommends it to me, be that a blogger, a journalist, or a friend. The thought of regularly spending time and money based on the newest number on a website just doesn’t compute in my mind. If I ever get so absolutely desperate for new games to play and can’t find anyone who can tell me anything about them then maybe I would resort for something like Metacritic. But I’m already over my head in games anyway. I’m learning about new games almost every day and I have plenty that I’ve picked up and never got around to playing.
Metacritic is a symptom, not a problem. It will change when we change. Michael’s central message is a good one: don’t use Metacritic. That’s a message I can get behind. I find little value in its scores, and even less in the lifestyle of a Metacritic addict. If you find yourself needing to consult Metacritic regularly then I would recommend taking a brief reflection on what exactly games mean to you, what value they have for you, if they are worth the arbitrary number on a website.
(As a side note, I hope I don’t seem like I’m attacking Michael Abbott. His blog lives up to its name, Brainy Gamer, and it’s worth checking out if you haven’t yet. I’m just picking on him for a little bit as I use his post as an example to make my point.)
Comment posted by Michael Abbott on May 09, 2010 08:13 PM UTC (permalink to this comment)
Hi John. Thanks for reading and responding to my post. I've long admired your work here, so I especially appreciate your thoughts.
I can't disagree with anything you've written here, but I do want to clarify a couple of things. You mention "I don’t fully understand what all of our anger is directed towards exactly." I'm honestly not angry about Metacritic, and I understand it functions as a useful tool for a lot of people. I began thinking about the subject when it became clear to me how many of my students rely on it to guide their purchasing decisions, but how few really understand how Metacritic works or what their ratings mean (or what they say they mean, anyway). So most of my attention in the post was focused on pointing out a few things that I find troubling about their methodology for turning reviews into numbers.
You're right, of course, when you say "Duh!" about my advice to rely on reviewers you trust. But here again I find many of the people I talk to (mostly students, but not all) don't tend to discriminate in that way. Metacritic essentially argues that the wisdom of the individual or small collection of voices is less reliable than the voice of the crowd. Busy game consumers who may not have time to sift through individual reviews or identify reviewers they trust are likely to find Metacritic's argument persuasive and its service handy and useful.
I think those of us who write and think about games all the time sometimes forget how disconnected we are from the vast number of players who probably can't name a single game critic or reviewer and have no inclination to figure out whom they can trust, aside from a personal friend. I want those people to better understand how Metacritic works...but they're probably not reading my blog, so, um, yeah. Well, you read it and took the time to respond, so maybe it wasn't a waste of time after all. ;-)
Thanks for the kind words about BG. Much appreciated.
Comment posted by John on May 10, 2010 01:47 AM UTC (permalink to this comment)
Thanks for reading and responding! I quite enjoyed reading your original post, so don't think I'm trying to criticize you directly. My response was more directed at the gaming culture and population in general. The thoughts behind this have been in my head for a while, but only came into words after I was inspired by reading your blog. I simply used you and your post as an example to make my point, perhaps unfairly.
I agree with you on the idea that holding Metacritic under the critical spotlight is probably a good thing, since it will hopefully dissuade people from using it. I simply don't want Metacritic to become a scapegoat for the more real problems that the industry faces.
Comment posted by Nick Gad on May 25, 2010 03:43 AM UTC (permalink to this comment)
Metacritic is virtually useless in Australia. The two major games chains, "EB Games" and "Game" have a 7 day money back policy, so if you go over the 7 days you can do a hassle free trade-in and get roughly half the purchase price back. The main problem is game genres! I prefer games in particular genres (RPGs) however modern games don't specify what type of game it is (hasn't happened since the Sega Master System boxes from my memory) and two recent examples come to mind: 1) Final Fantasy series has traditionally been an RPG, however its latest instalment (FF13) is quite a luke warm RPG. 2) The makers of Brutal Legend, advertised Tim Schaffer's involvement at every opportunity but purposefully omitted that the game is a console RTS (which have ALWAYS been crap due to limitations of controllers). So in essence the problem is not Metacritic, game stores in Australia allow you to try out a game for free, its game developers and publishes who invest millions, develop a dud game and then try to flog it off by buying reviews (Kane & Lynch) or having hyped marketing with little actual game details. Th only problem with metacritic is that its owned by Gamespot, and that's a conflict of interest plus they regularly block user reviews!
Post your response