Score Inflation in the IPA Category

Eno Sarris, November 08, 2016

I don't know if I'm getting better beer, but I've found myself filling in many more four-star reviews recently than I remember.

I mean -- I could be getting better beer. I'm trading for beer, and I've developed a database of beers in my head and on this website that help me avoid the clunkers. As craft beer slows a bit, maybe the less quality brewers are falling away. I like to drink beers that are recommended to me by a human being, too, which cuts beers off the low end. 

There are still times when I'm flying blind. This past weekend, I was having beers with friends that didn't want to see me jump on my phone and start searching for the best option. I ended up with a Mudshark IPA and gave it fewer than three stars. Phew. I'm not contributing to grade inflation. Possibly. 

But it does look like there's been some of that. Scores for the combined mega-IPA category (American IPA, IPA - American, and Imperial Pale Ale) have been steadily increasing over the last three years:

The effect is modest because we're really talking about the average IPA moving from around 3.4 to 3.5, or maybe even a smaller effect. But it's fairly steady, and backed by the largest sample we can provide, usually more than three thousand checkins a day. 

Showing something like this might be happening is a lot easier than explaining why. 

We could be making better beer! Or beer rating site users could be finding better beer, thanks to the apps themselves, or other websites. Maybe drinkers aren't checking into bad beers as often. 

That last one is the most interesting. What we're tracking here are the years in which the social check-in apps for beer have become more popular and prevalent. Over the course of these last three years, drinking craft beer has become more of a social media phenomenon. Drinkers have publicized their beverages more.

Sorta makes you wonder if the act of publishing a grade makes you more likely to inflate the grade, even subconciously. "This is pretty good, maybe a little above average" can easily become "This is great, people" if there's an audience. Maybe "This is just slightly worse than average" just becomes a non-rating. 

If this is really the case, it kinda means we're getting worse at rating beers.

The Audience Effect says that merely having an audience changes the way we perform certain tasks. The Yerkes-Dodson law tries to explain the variance in performance due to the audience and says that we get a little better at practiced skills and worse at more difficult tasks. I suppose you could call rating a beer a practiced skill, but considering the fact that people are trained (often very rigorously) in the art of rating beer, it sounds more like a difficult task. 

I love tracking my beers. I input almost every new beer into untappd, and it helps me remember which beers were amazing, and also see other amazing beers people are drinking. It would be a shame if the simple process of logging onto untappd was making me worse at rating beers. 

Maybe beers just are getting better, then. That's what I'll go with.