Quantcast

Checking New Release Ratings Against Their Established Value

Michael Donato, February 19, 2014 -   

Hype has become a huge part of craft beer. There are a couple dozen awesome beers that come out once a year that many fans look forward to; Beers like Bourbon County Brand Stout or Hopslam. You can read Larry Koestler's take on vintages and the 'this was better last year' effect. 

Does anticipation of these beers lead to better ratings?  Often these beers reach more and more people with each year, as the hype leads to the brewery looking to distribute to as many people as possible.  Do the new drinkers agree with the hardcore fans about how awesome a beer is? 

Troegs Nugget Nectar has a 9.21 BAR and is 26th on our leaderboards. It's a delicious beer that's desired by many and there have already been many BeerGraphs posts about its release. This year's version was officially released on January 6th, and I'm going to use that as a pivot point and take a look at the ratings before and after that date. There will be some bleed over, but that should be minimal. So far, this year's Nugget Nectar is not doing as well as it has previously. 

Nugget Nectar

Rating

Check-Ins

New Release

4.332586

2231

Previous

4.407412

14894

 You'll see that the new release is rating about .075 lower. That seems trivial, but it's really not. That's the equivalence of 334 people giving it 4.5 stars instead of 5, or 15% of it's drinkers. 2231 is not a trivial amount of check-ins, but it is about one seventh of the data we have for the previous set so maybe the rating will eventually even out. I wouldn't bank on that though; beer, particularly hoppy beers, are best fresh and as time passes more and more of the beer is going to sit in hot stores or in the sun or simply just not age well. 

Let's repeat the experiment for Bell's Hopslam, fifth on the leaderboards with a 11.81 BAR. I'm going to combine all of the various vintages we find on Untappd. The recipe doesn't change, so besides age 2014 is the same as 2013 is the same as checking into the non-vintage entry. This will still let some older beers that have been hoarded into the new release group, but only a few. 

Hopslam

Rating

Check-Ins

Previous

4.580846

33137

New Release

4.506212

5634

Besides volume, the numbers are pretty similar. Hopslam's new release is rating about .075 lower than it was previously. The same 15% of drinkers are rating it about a half star lower, me among them, only with the volume difference this time it's 833 people. Again I would think 5634 would be enough check-ins to get a stable sample, but perhaps more people try a beer when it just comes out, especially one with as much hype as Hopslam, and after that only the fans that are likely to continue rating it a 5 continue drinking in and the rating then comes up a little. I've checked these numbers a couple of times over the past week though, and each time the gap has been bigger. 

Let's take a look at a third beer; Lagunitas Sucks. Sucks has an extra month of data in the new release group since it started appearing again in December and with this beer at least that makes a difference.

Lagunitas Sucks

Rating

Check-Ins

Previous

4.292588

25365

New Release

4.285902

10299

This is much closer, only a .007 difference, and the samples are much closer in size than the other two beers. Is this a result of the extra month of data? Let's try taking out the last four weeks.

Lagunitas Sucks

Rating

Check-Ins

Previous

4.288457

33024

New Release

4.317373

2642

Quite the opposite effect here. The new release of Lagunitas Sucks actually is rated better than the older versions. Thats 153, about 6%, of drinkers rating the new version a half star higher. This bucks the trend that we started to see a little bit.

Beer can be a very intimate experience. When you drink a beer can change your opinion of it. Where you were when you had it matters too; a crowded bar while your sports team loses provides a different emotional state than an analytical tasting at home. If you're having a new beer for the first time that a lot of you friends have recommended and hyped you might go into it with a different expectation that colors your opinion of the beer differently than if you're picking up a new release of an old favorite and greeting it like an old friend.

New releases are always an exciting time, and while they might not be consistent across all beers they're often treated differently. I'd like to check back on these three beers in another month or two and see how the new releases have responded as the ratings level out. 

Michael drank all three of these beers both in the last two weeks and last year. You can follow him on Twitter and Untappd.

comments powered by Disqus