HomeFeatures
Unravelling the magic and alchemy of MetacriticScore wars
Score wars
Image credit:Metacritic
Image credit:Metacritic
On the latest-but-one revision of theirAbout page, Metacritic describe the process of calculating a videogame’s aggregate “Metascore” as a kind of “magic”. The FAQ cheekily invites you to “peek behind the curtain”, evoking the figure of the theatre conjurer beckoning the audience on-stage to inspect the props, before performing the trick. You’re only shown so much, however. There are tables for conversions between different review scoring systems, demonstrating how a B- becomes 67/100, but the “weighting” Metacritic gives to each source publication when producing the combined Metascore is a closely-guarded mystery.
Metacritic’s founders are former attorneys, and one way of analysing the “magic” of review scores and Metascores is through the lens of copyright law. Cornell professor James Grimmelmann digs into this in a - wait, come back! -genuinely very readablepaper about whether scores and ratings can be copyrighted in the US, which you candownload for free over thisaway. Combing a brace of court cases from the past century, Grimmelmann slowly puts together a picture of the product review score as a kind of shape-shifting, supernatural trinket - a strange, evolving compound of fact, opinion and “self-fulfilling prophecy”.
Image credit:Deconstructeam/Devolver Digital
We call Metacritic an “aggregator” but you won’t find the verb “aggregate” on the site itself - after all, if “metamagic” were just about adding scores up, it would be harder to define as “proprietary”. On their older About page, Metacritic present themselves instead as “distilling” critical verdicts, so as to “capture the essence” of critical opinion, like a medieval potion-brewer attempting to refine a substance to its purest form.
As any reviewer could tell you, Metacritic’s “distilling” of critical voices very easily becomes distortion. One intrinsic failure is that the site’s editors treat every other numerical scoring system as a derivation of their own scale, so that a 4 out of 5 star rating is expanded into 80% on Metacritic, whereas in practice, these two grades conjure very different emotions, based on the ways they’ve been applied across media generally. Another problem is the practice of quoting a single paragraph or sentence from each review, which leads to misreadings and arguments based on a single phrase out of context.
The ostensible light at the end of the tunnel here is Metacritic’s rival OpenCritic, launched in 2015, which doesn’t “weight” individual reviews the same way. But OpenCritic operates its own school of non-replicable “magic” born of smaller forms of opacity, such as what exactly distinguishes a “Top” from a rank-and-file reviewer when calculating the “Top Critic Average”. And it too “distils”, conflating score systems and cherry-picking paragraphs, in order to devise its product.
It’s perennially hard to say how much Metacritic and other aggretators really affect the fortunes of game developers. Different games, types of game and studios have differing levels of exposure to aggregator culture, with canny or lucky developers building audiences outside the usual promotional channels. But it certainly has clout. The industry’s history is awash with anecdotes about teams being denied bonuses and missing out on gigs because their latest work fell short of that lush green 90. This is Grimmelmann’s idea of the score as a “self-fulfilling prophecy” in action: the aggregate is an oracle that decides what will sell and who is worth banking on. But Metacritic doesn’t just influence what gets made - it also helps decide what gets preserved. In 2008, Microsoftannounced plans to delist a number of Xbox Live Arcade games based on their Metascores, as though paving over the fossil record at a certain level.
I don’t think videogame review aggregators have nearly that level of influence, nor do I personally know of any instances of outright score-fiddling at places like Metacritic. But there’s a comparable relationship between review aggregators and the industry and field of gaming, and it isn’t hard to find precedents for how these playfully shrouded systems might be abused - a few weeks ago, Vultureunearthed evidenceof third-party score manipulation at movie aggregator Rotten Tomatoes.
All this may read like the usual Metacritic teardown from a long-suffering games journalist, and I must admit I’m very relieved to now be writing for a site that doesn’t do scores at all (spoiler alert: an earlier draft of this essay was penned in the wilds of freelance). The thing is, even as a reviewer who is used to getting it in the neck from outraged Metascore-chasers, I enjoy the magic and alchemy of “metacriticism”. I, too, covet the lustre of a Metascore page that is solidly green, cleansed of yellow or red contaminants, much as I enjoy looking at a Destiny inventory’s worth of purple weapons. I also like aspects of the discussion around Metascores - it’s fun to dip into aggregator prediction threads and compare player speculations with my own, under-embargo opinion of a forthcoming game. And I think review aggregation can have genuine utility within the undead machinery of commodity consumption, if only as a way of saving labour. At their best and humblest, aggregators, like scoring systems themselves, offer simple shortcuts for people who don’t have the time or interest to read several pieces of criticism. They exist to give you a rough overall sense of the huge volume of criticism out there, while directing readers outward to individual critics.
