Should Distribution Area Factor Into BAR?

Matt Murphy, July 02, 2013 -   

When we unveiled Beers Above Replacement, one of the common concerns raised was that the equation did not account for distribution area. For example, Three Floyds' Zombie Dust came up as the number one American Pale Ale (APA) by BAR, which was driven not only by having the highest wOBAR among APAs, but also by ranking fifth in number of check-ins. Zombie Dust received almost as many check-ins as Boston Beer Company's New Albion Ale, and ahead of Kona's Fire Rock Pale Ale. As Eno Sarris discussed in his search for national and regional replacement level beers, the latter two beers are readily available throughout the country. Meanwhile, Three Floyds' distribution is limited to its home state of Indiana and the immediately surrounding states. This is illustrated on seekabrew.com:

On one hand, sticking strictly to number of check-ins makes sense. We want BAR to reflect the total "value" that a beer has given craft beer drinkers (specifically, Untappd users), so it shouldn't make a difference where those check-ins occur. However, it also makes sense that a beer's value should reflect the total number of beer drinkers that have an opportunity to enjoy it. While there is still room for debate regarding the different factors that should be included in BAR, today I want to focus on different ways to look at distribution area, should we decide to include it.

The first dilemma is whether distribution should be calculated for each brewery or for individual beers. Here are a few beers from Sierra Nevada, one of the few breweries with national distribution, along with the median distance between the check-in location and the brewer:

Beer Median Distance to Brewery
Pale Ale 1927
Ruthless Rye IPA 2057
Two-Headed Ruthless Rye 139
Northern Hemisphere Harvest Wet Hop Ale 1700

This chart illustrates a fact that we already know: different beers from the same brewery can have drastically different distribution profiles. Three of these four Sierra Nevada brews are distributed nationally, with half of the check-ins occurring at least 1700 miles away from the brewery. Therefore, if distribution area is going to be factored into BAR, it needs to be calculated for each individual beer, rather than the brewery as a whole.

So what are some options for measuring distribution of individual beers? Recently, I wrote about the differences between ratings for in-state and out-of-state check-ins. Sticking with this theme, one possibility would be using check-in location to approximate what percentage of a given beer is consumed outside of the state where it was brewed. Let's take a look at a few well known beers:

Beer % Out of State
Dogfish Head 60 Minute IPA 96.0%
Three Floyds Zombie Dust 79.7%
Sierra Nevada Pale Ale 77.6%
Russian River Pliny the Elder 36.8%
Russian River Pliny the Younger 8.6%

Some of these numbers seem to make sense. Sierra Nevada Pale Ale (SNPA) is one of the most widely distributed beers in the country, and over three quarters of its check-ins come from outside of its home state of California. Meanwhile, Pliny the Elder has a much lower percentage of out-of-state check-ins, as Russian River has more limited distribution, while Pliny the Younger (which is draught-only) is even tougher to find outside of California.

However, 60 Minute IPA and Zombie Dust raise some red flags. The small size of Delaware means that the vast majority of Dogfish Head's beer is consumed outside of its home state. This gives it the highest percentage of out-of-state check-ins from the five examples, although we know that SNPA is the most widely distributed. A quick look at median distance-to-brewery shows us that half of the check-ins for 60 Minute IPA occur over 274 miles away from the Dogfish Head brewery, and half of the Zombie Dust check-ins are at least 127 miles from Three Floyds. This number pales in comparison to SNPA (1927 miles) and is even less than Pliny the Elder (424 miles). The size of the home state plays too great a role in this metric, so we need to look elsewhere.

Mean and median distance to the brewery could prove to be a good measure of beer distribution. Here are the numbers for the same five example beers:

Beer Mean Distance to Brewery Median Distance to Brewery
Dogfish Head 60 Minute IPA 464 274
Three Floyds Zombie Dust 304 127
Sierra Nevada Pale Ale 1593 1927
Russian River Pliny the Elder 793 424
Russian River Pliny the Younger 263 63

We see the same general trends using mean or median distance, but there are some noticeable differences between the two. SNPA is the one beer with true national distribution, and is also the one beer where the median distance is greater than the mean distance. While the majority of Zombie Dust check-ins are in the Midwest, its high rating means that beer geeks across the country want to get their hands on it. This results in a large number of check-ins from states such as California and Florida, presumably through beer trading, traveling, or special events. While much of Dogfish Head's distribution is focused on the Atlantic coast, they also ship to the west coast, which explains the discrepancy. Russian River is a sort of hybrid, as they distribute a small amount of their beer to Pennsylvania, and the Pliny hype makes it a popular target for trades (or in the case of the Younger, unauthorized hand-filled bottles being sold on eBay).

There are two important caveats that hurt this metric as a proxy for distribution. The first is breweries that have a relatively small distribution area that is not very close to the brewery itself. For example, imagine two breweries in New York State. One is located within New York City, and another is located 50 miles outside of the city. Each brewery produces a beer that has 90% of its check-ins within NYC. Even though the distribution profiles are nearly identical, the average distance to the brewery will be quite different. (This is obviously an extreme example, and determining the actual effect of this phenomenon on distance-to-brewery would require further investigation.)

A second issue is international distribution. All of the numbers above are limited to check-ins within the United States. International distribution would have a drastic impact on mean distance-to-brewery that could completely skew the numbers, although using median distance could prevent this. However, measuring distribution of beers from outside the US could be troublesome. Since the vast majority of the Untappd userbase is in the US, the average distance-to-brewery would be off the charts for nearly every foreign beer. This would require foreign beers to be treated differently, and if you can't factor in distribution the same way for every beer, then is it even worth doing?

I'll save these questions for another time and take a look at how distribution area based on median distance-to-brewery would affect the ratings of some American Pale Ales. The median distance-to-brewery for all APAs is 465 miles. Since we don't know how accurate of a measure of distribution this is, we don't want it to influence BAR very heavily. Here's a rough equation I came up with:

Distribution Factor = [ 6 + ( median distance-to-brewery / 465 ) ] / 7

This means that a beer that is only consumed at the brewery (0 miles) will have a 14% reduction in BAR compared to a beer with average distribution (465 miles). Meanwhile, a beer like SNPA with true national distribution would get a boost of around 45%. Here are each of the top-15 APAs with at least 100 check-in locations and how they would be affected by this new metric:

Beer Brewery BAR Median Distance to Brewery BAR w/ Distribution Difference
Zombie Dust Three Floyds Brewing Company 14.45 127 12.95 -1.5
Edward Hill Farmstead Brewery 8.72 56 7.62 -1.1
Dale's Pale Ale Oskar Blues Brewery 6.01 1235 7.43 1.42
Red Chair NWPA Deschutes Brewery 6 1191 7.34 1.34
Simply Simcoe Fat Head's Brewery 5.9   5.06 -0.84
Moon Man New Glarus Brewing Company 5.1 93 4.52 -0.58
Pale Ale Sierra Nevada 4.85 1927 7.03 2.18

What do we see? Beers with better distribution from breweries such as Sierra Nevada and Oskar Blues get a significant bump in BAR, while those that focus more on supplying the local and regional market suffer. If the point of BAR is to showcase beers that are both excellent and widely available throughout the country, then this seems to do a good job. However, it feels to me like this is just penalizing the small breweries that manage to achieve moderate to high check-in volume despite relatively small distribution areas. This chart also raises the concern that not even half of the top-15 APAs have a significant number of check-ins that contain location information.

While some sort of distribution area or "range" metric provides us with valuable information, I'm not convinced that it has a place in BAR (or at least that its contribution should be small relative to other factors). In baseball, when a player's value is calculated, it doesn't matter whether his contributions came at his home park or on the road. Do you think that it should be different when measuring the value of a beer?

When he's not writing about beer, Matt works at the NYU Medical Center where he does cancer and stem cell biology research. You can find him on Twitter at @murphym45.

comments powered by Disqus