Everybody has a few favorite breweries. Maybe it’s the brewery that produced the first beer you tried in college that opened up your eyes to a world beyond the adjunct lager. Maybe it’s a brewpub in the town you grew up in, or a local brewery in the area where you live now. Whatever the reason, we may not be completely rational when it comes to our favorite breweries, and we might give them a friendly bump (whether conscious or subconscious) when we’re rating one of their beers.
When I first started looking at the Untappd data we use at Beergraphs, one of the topics that most interested me was how these types of biases can affect beer ratings. I’ve lived in the state of New York for my entire life, so a lot of my favorite breweries are in NY state. I’ll admit that when I’m rating a beer, I’m probably more likely to give it the benefit of the doubt if I know that it was brewed in my home state. While I’m certain that I’m not the only person who does this, I wanted to see if this bias shows up in the data, and whether we can quantify it.
One important caveat is that while we have the data for where many of the ratings were made, we don’t know the hometown or state of the user. However, I personally believe that the majority of check-ins on Untappd occur relatively close to the user’s current city or town of residence, and probably in the same state. There will likely be a significant amount of noise due to beer tourism and travel, but there isn’t a good way around that (yet).
So, the real question here is, “Are Untappd users more likely to give a beer a favorable rating when they check-in within the state where the beer was brewed?” I had over 350,000 check-ins with location data to work with.
As I expected, the ratings for in-state beers were higher than those for out-of-state beers. However, the difference was not very large. On average, in-state beers were favored by an average score of 3.23 to 3.17 for out-of-state beers, for a difference in 0.64 points (on a five-point scale). While Untappd recently introduced a half-point rating system, the data we have is for whole numbers only. So what this really means is that for every 16 people who taste and rate a beer, one person who is in the state where the beer was brewed will give it an extra point. It may not seem like a lot, but over thousands of check-ins, little differences like this add up.
When I saw this, I was curious about whether check-ins at or in the immediate vicinity of the brewery or brewpub were driving this trend. My guess was that if you’re actually at the brewery, or in the town or city where it was brewed, you might be more likely to give it a little boost. Maybe the beer is more fresh, or maybe there’s just something about being right there at the brewery when you’re drinking the beer that makes the experience more enjoyable.
We use the geolocation data to calculate the distance between the beer rater and the brewery where the beer was produced and then store that number in our database. To my surprise, ratings that were made at or within 10 miles of the brewery received slightly below-average scores (3.17). Once you get more than 10 miles away from the brewery, the average rating jumps up to 3.26. This level is roughly maintained up to a distance of 500 miles from the brewery, after which it sees a sharp decline all the way to 3.04 (750-1250 miles from the brewery). Lastly, check-ins more than 2000 miles from the brewery actually had the highest rating of any distance group (3.29).
If Untappd users give higher ratings to in-state beers, why doesn’t the same benefit apply when they are drinking the beer at or near the brewery? My first guess was that people who account for the majority of check-ins at or near the brewery are locals, and when you can get a beer whenever you want, it becomes boring. People are excited by beers that are rare, beers that they might not have access to on a regular basis.
A second theory could be that there are a lot of small breweries and brewpubs that just aren’t that great. Breweries that produce excellent beer are going to make money and be able to expand their distribution. Think Sierra Nevada, Lagunitas, and Dogfish Head. The increased distribution means that the vast majority of check-ins for these beers will be miles away from the brewery. Since a brewery can only have so much traffic at their tasting room or brewpub, the smaller breweries and brewpubs will be overrepresented in the check-ins less than a mile away. While there are plenty of great small breweries, there are also a lot of mediocre ones, and they’re competing with some well-established breweries that have been making good beer for a long time.
Whatever the reason, Untappd users seem to care more about the state listed on the beer label than they do about how close they are to the brewery. While the overall difference is rather small, over a large number of check-ins, these differences can have a significant effect on BAR. Interestingly, there are some individual states where the gap between in-state and out-of-state ratings is much greater, and a few where the trend is actually reversed. That, however, is a topic for another day.
When he's not writing about beer, Matt works at the NYU Medical Center where he does cancer and stem cell biology research. You can find him on Twitter at @murphym45.