The replacement level beer. It's important that we nail it down. It's the beer that we're comfortable with, the worst-case scenario beer, the beer you know you can get when you head into your local gas station beer section. If another beer can't do much better than that, it deserves to have a negative sign out in front of its rating, right?
Back when we were looking at regional replacement level, we had a hard time deciding where we should cut the sample. Should we look at the top 60% of volume within the style? The top 70%? Which cutoff gave us the best sample of beers that were readily available to people? We resorted to the smell test to settle around 70%. Grab the top 70% by check-in quantity, find the worst beer, and that was replacement level: the worst beer that was readily available to most people in the region.
That's all fine and good, but can we see what a style looks like? If we ranked beers from the most popular to the least, is there a moment when it's obvious you've left the 'readily available' beers behind? Can we chop the tail -- all those beers with five, ten, and twenty check ins -- off the main body of a style in order to find the replacement-level beer?
Let's see. Here are the pale ales, ranked by count in our database. Is there a tiering that immediately becomes obvious?
First -- that's really top-heavy. The top five beers alone (Sierra Nevada Pale Ale, Oskar Blues Dale's Pale Ale, Three Floyds Zombie Dust, Mirror Pond Pale Ale and Widmer Brothers Alchemy Ale) make up almost 15% of the entire sample. The top ten make up 20%.
Let's look at the style with the top ten taken out. Now we're looking at the bottom 80% of the style.
Similar looking, in that there's a long tail of little beers reaching out into infinity. That's the tail we'd like to cut off -- the beers that a few people have access to, beers that could *not* by definition be replacement level beers. Let me annotate this graph with a few places where it looks like there's some tiering. These are possible replacement level cut-offs. It's important not to use minimums (like, say 1000 checkins) because our sample is growing with every day. Better to use a percentage.
The annotated shape of the pale ales:
There are changes in the slope of the line at these points. Let's see what they represent.
The first is around 750 check-ins, and if you sum the counts before that moment, it's 60%. There's obviously a differencein the slope of the line there, so 60% looks like a defensible number. The smallest beers in that grouping are Citra from Knee Deep, Baltimore Pale Ale from Full Tilt, Pale Ale from Saint Archer, and Amarillo Pale Ale from Hangar 24. That's a decent selection of beers from around the country that are big enough to be considered for replacement level.
That next line? Amazingly, it represents 75% of the total volume. A full 25% of beers in this style have fewer than 240 check-ins.
If you use 75% as your replacement level grouping, you let some smaller beers into the sample. Organ Grinder from Mankato Brewing, Hopper Pale Ale from Madison River, Cutthroat Pale Ale from Tree Brewing, Tulsa Rugby Ale from Prairie Artisan, and Mt. Tam Pale Ale from Marin Brewing -- these are the beers at the bottom of the sample if we are to use 75%.
It looks like there are decent visually-based arguments for the 60% and 75% cutoffs. At both moments, the shape of the pale ale style changes, and the slope of the line flattens.
Once again, though, we are reduced to the sniff test. Should beers as small as Mt. Tam Pale Ale be part of our replacement level sample? Or should we cut the sample at 60%, even if it raises replacement level some?
Even if we end in questions, it feels good to know that, at least within the beer with the biggest sample in all of craft beer, we weren't far off. Things change at 60% and 75%, and now we just need to nail that last bit down to make sure we've got the right replacement level.