Another of my new Gamasutra/ GameSetWatch columns bellyflops onto the internet. It’s a rewritten and extended version of some thoughts I wrote here last year on review scores, a subject that deserves close attention in the light of Microsoft’s decision to use Metacritic scores to decide which games should get dropped from their Xbox Live Arcade service. Hopefully it’s clear, concise and with just a pinch of the polemic to annoy the right kind of readers.

Last month a British games journalist reviewed Xbox Live Arcade’s Penny Arcade Adventures for two different publications. In one of the magazines the game scored 4/10 while, in the other it was awarded 68%. While it’s a discrepancy that caused some to raise their eyebrows, most commentators acknowledge that the difference simply reflects each publication’s own particular use of the numerical review scale.

Two weeks later Microsoft announced their plans to remove games with an average Metacritic score of 65% or lower from their XBLA service. If the decision on whether to keep Penny Arcade Adventures on the service were to be based solely on the judgement of this reviewer, its fate would swing on which review was looked at.

While a game’s Metacritic or Gameranking average score has often been used to dictate the size of a development staff’s bonuses, EA’s decision to use numerical scores as the criterion for has elevated the numbers issue a whole new level of consequence.

Some argue that scores represent different things to different publications, one title’s 4/10 being another’s 68%. Others question why, when scores rarely tally with a game’s commercial success, we should use them to make commercial decisions? Always, the question behind the question is: do review scores actually matter and, if so what do they even mean?

At a glance, review scores seem to be the most harmless of things. While good critics will bemoan having to reduce a 1000-word piece of incisive criticism to a number on a 10 point scale (or, um, 19 point scale if you’re GameSpot), to the average consumer they offer a useful shorthand reference point with which to compare different titles and inform buying decisions.

You can read the rest over at Gamasutra here.