Friday, October 28, 2005

RT vs. MC

Rotten Tomatoes is a great idea. Take reviews from lots of sources, rate whether each review is overall positive or negative, and then give the movie a score based on what percentage of its reviews are positive. Then give it an overall rating of "fresh" or "rotten" depending on whether its percentage exceeds a certain threshold.

The problem is that two movies can have the exact same percentage -- say, 40% -- and yet the reasons they each got 40% can be radically different. Maybe Film A inspired joy and excitement in 40% of the reviewers who saw it, and the other 60% thought it was a bag of flaming dogshit. Maybe Film B elicited shrugs from every reviewer, 40% of whom just barely gave it a positive review, the other 60% of whom barely gave it a negative review. And of course there's the problem of the meta-reviewer having to decide at what point a review crosses the line from positive to negative.

Metacritic takes a slightly different approach. Each review is given a score from 0-100, depending on how positive it is. The scores are then averaged into a final percentage rating for the movie. In practical terms, this leads toward a narrower distribution of ratings; high-scoring movies on RT don't tend to score as high on MC; and low-scoring movies on RT don't tend to score as low on MT.

MT's approach is more accurate; it gives a more precise picture of the aggregate critical reaction. You still have to look at the details of each movie to see exactly why it got rated the way it did. If Metacritic thought enough people knew what a standard deviation was, they'd probably include it.

Both sites attempt to turn subjective opinion into objective fact; but they both still require their meta-reviewers to, well, review the reviews, and those reviews are of course subjective. I don't think they're useful for much more than entertainment, or maybe helping decide which flick to blow your paycheck on this weekend, but a few times I've seen people quote numbers from one site or the other as if that proves that Movie A is better than Movie B in some truly objective sense. Meh.