I was talking with someone smarter than I am about normalizing beer ratings from region to region and translating the replacement level concept to beer. They said something very simple that blew my mind about a similar project they were doing. Why not look at national beers and how they fare in each region?
By accounting for the local tastes in each region, we'd be accounting for replacement level in an end-around. For instance, if you're in California and you are drinking a Sierra Nevada Pale Ale, you'd be forgiven for thinking it's replacement level. You have a lot of great West Coast pale ales at your disposal. In another state, you might rate it higher. By looking at these differences, we can find a way to talk about replacement level from a truly even playing field: ratings on the same beer in different markets.
So, who likes Sierra Nevada the best? Here are the average ratings by state in our database:
South Dakota, come on down. You don't have many breweries in your state... but, at the same time, you're not the wasteland that some states in the south are. Take a look at the header image, which shows per capita breweries by state thanks to wikipedia. South Dakota is mostly lacking in... people. And so, yeah, the sample strength isn't that high on the South Dakota number.
You have to go down to Hawaii to get more than 150 check-ins in our database. Remember, we only go back to early last year, and we don't get every single check-in from untappd's database, as the API limits us to some extent.
But look at Hawaii. 150+ people consider Sierra Nevada Pale Ale a 3.67-star beer. And over 200 people in Missouri think that it's barely a three-star beer. To some extent, that reflects two things, most likely: the number of breweries in each state, and the number of available breweries in each state. That's exactly what replacement level should be trying to account for.
Now there are the caveats: We know that distance from the brewery has a few effects. For one, there's the question of freshness. But the truly national brands have done a good job of thinking through their supply chains and making sure their beer is fresh, we'd hope.
Then there's bias. There seems to be a slight home field advantage for close-to-home breweries, and then a large jump for Whales -- beers that travel very far and are therefore coveted and rewarded for their troubles. Read Matt Murphy's excellent breakdown of the phenomenon. There's a whiff of whale-ism for every beer in Hawaii.
But do we need to correct for this? Do beers have a level playing field in each state (a Whale is a Whale for everyone?) How do we correct for this if we wanted to? How much should we correct for freshness concerns (especially when it comes to international check-ins)?
Those are just the background questions. The main question is:
Do we really want to rinse every pale ale check-in coming out of Hawaii for a state-wide bias based on how the state has rated Sierra Nevada Pale Ale when compared to other states? Should this be BAR or a new stat?
Please. Please tell me what you think. Someone smarter than me came up with this and so I need you, people that are smarter than me, to help me make sense of it.
Interested in Daily Fantasy Baseball? Use this link to try out Draftstreet with a special offer -- they'll match your deposit and give you a low-cost entry into a game this week.