Tuesday 8 December 2015

Using beer ratings as market research

I've been working a lot lately on firming up the first 10 beers I plan to brew at Elusive, drawing from what is quite a large collection of recipes. This got me thinking about market research and ascertaining how well these beers might be received. Although that's not necessarily an indicator of potential sales, I figured it might provide useful input into developing an overall strategy. In an increasingly crowded market, simply starting up and hoping for the best is a risky approach. There's no longer a 'build it and they will come' road to market. Whilst you might shift your first few batches out of curiosity, if they aren't well received at point of sale, it'll get tougher from there.

Like many beer drinkers, I use Untappd (when I remember!) to record the beers I've drunk and give a rating. Personally, it's mostly used as an aide memoire but others like the gamification aspect of collecting badges and the other social features it provides. Untappd is a great source of data, although compared to sites such as ratebeer, the data is perhaps less 'clean' given the comparative lack of moderation - for example, if the first drinker of a specific new beer adds it in the wrong style category, it may never get changed. With that in mind, the analysis here should not be considered useful for anything more than satisfying my curiosity! There were some other boundaries I set when analysing this data:

  • A style would be discounted if there were fewer than five beers listed
  • Data would not be 'cleaned' at all
  • Since Untappd breaks down by country, I'd focus just on England
  • Overseas rates of English beers would be included (don't know how to exclude them)

Before we get into the data and findings, it's useful to have a read of Untappd's explanation how scores are determined. The first thing I looked at was the top rated breweries. This is readily available in the user interface:


There are a couple of standouts (for me) here - the first being Hanlons and the second being Samuel Smith's. Hanlons used to brew the much sought after Thomas Hardy's ale. A quick look at their ratings shows this is a common and inevitably highly rated check in, mostly from overseas. Is this driven by scarcity? Possibly. Sam Smith's stands out because of the huge number of ratings. Again, this appears to be due to a strong overseas influence. On to the top 15 rated beers overall, we see a pattern starting to emerge:


If we discount our Sam Smith's outlier, these are mostly strong beers. There's not a huge delta in the number of ratings (relative to the outlier) but the average ABV is just under 10%. This got me thinking about skew by style. That is, if you look at the top rated beers by style, how much variance is there in the average. The graph below (click to expand) shows that skew. It was created by taking the top 5 beers in each style and averaging the scores, then comparing that to the overall average across them all, which was a score of 3.70.


Here we can see a very positive skew towards the 'big' Stouts and IPAs, as reflected in the overall highest rated beers. In analysing the data to create the above graph, I noticed that in some cases there was also a large(r) delta within the styles too. That is, the top rated beer by style was varying degrees higher than the average of the top 5. Those were often the least popular styles (e.g. Cream Ale) but in some cases were also popular, suggesting an 'opportunity' to score relatively well (because of the positive skew) compared to brewing a beer with a lower overall average across the top beers in style. I can't think of a way to show this graphically but these styles stand out as 'opportunity' styles:
  • Imperial Stout
  • Milk Stout
  • Strong Ale (English)
  • Porter (English)
  • Mild (English)
  • Red Ale (American Amber/Red/Imperial)
At the other end of the scale, these styles could be considered 'opportunity' styles for other reasons, because they skew lower but have variance within the style, suggesting that attaining a high score relative to other beers in the style might be easier compared to the other end of the scale:
  • Chilli Beer
  • Bock
  • California Common
Finally, looking at all of the included styles and counting which breweries had the most number of top rated beers, we see that London's Kernel Brewery stands out by having the top rated beer across 9 different styles.


Does this make for useful market research to a would be brewery? Possibly not, but at least it satisfied my aim of dragging this blog back to where it started - with pretty graphs!

17 comments:

  1. So very pretty. Could the weighting towards strength actually be measuring price? As in, people are more likely to give a high rating to a beer they paid a lot for, which in turn is likely to be a strong one.

    ReplyDelete
    Replies
    1. That could well be the case - those beers can be more expensive. No way of analysing that here though, although I could look at the prices of specific beers (would take a while).

      Delete
    2. Definitely think some people's ratings are influenced by the price they pay, the abv and also the hype behind the beer. People just dont wana rate a big barrel aged stout they paid a lot of money for below a 4 even if they ddint think it was that good. But people will quite happily give a excellent pale wich they enjoyed more than the stout 3.5 because it's "just a pale ale"

      Delete
  2. A lot gets made of the tendency of beer ratings site top 50s to be dominated by Imperial Double Extra type beers and how this proves that people rating beers on them must all be adolescent extremophiles who have no understanding of subtlety. But isn't it a fairly natural thing that when asked to assign a number between zero and five to summarize the whole complex, subjective, multisensory experience of drinking a beer, a lot of people will tend to assign a bigger number to a more intense, full-on, attention-grabbing experience?

    FWIW, though, since they don't generally rate to style, this does mean that the easiest way to game them seems to be to take a fairly subtle and understated style and then ramp up the OG and dry hop the proverbial hell out of it.

    ReplyDelete
    Replies
    1. I agree there's likely a general tendency to rate bigger, bolder beers higher but there's certainly some interesting data outside of those top rated styles.

      Delete
  3. Really interesting read. Well done for putting it together. Ratebeer reviews tend to be a little more honest I feel.

    ReplyDelete
    Replies
    1. Thanks Wayne. I think people take more time over ratebeer reviews, plus of course the base data itself is closely moderated so there are fewer duplicate or miscategorised beers. I chose untappd because it's more likely to be 'first impressions'.

      Delete
  4. This comment has been removed by the author.

    ReplyDelete
  5. A lot of RateBeer scores are based on small measures sampled at tastings and festivals which means subtle beers don't necessarily get a fair crack of the whip. I know the hardcore raters will argue that their palates are developed enough to accommodate that but I'm not convinced. Having said that I'm about to do a tasting tonight with 30+ beers and I'll log my scores on Untappd so swings and roundabouts I guess.

    ReplyDelete
    Replies
    1. Thanks for the insight. Interesting point - I would guess that tastings are more common amongst ratebeer users than untappd, which perhaps aligns better with the casual drinker. I know some use both but I see ratebeer generally as a smaller, more hardcore set of record keepers (hence the tighter moderation & admin) than those solely on untappd.

      Delete
  6. As a user of Rate Beer and a user of Untappd there is a distinction that it's easier to give a higher score on Untappd as it's only out of five and there's no judging criteria such as smell, how it looks and how it is on the mouthfeel. This makes RateBeer a bit more complicated for the generic users. Untappd is very accessible and easy. They have also absolutely nailed the rewards' system so more people use it as they feel inclined to do so. Do this get a reward. Repeat. Happy consumer. Rate Beer is a lot more technical, takes up a lot more time. I can drink a lot more than I rate so if I'm at a festival I switch more to Untappd. More so that I get anxious if I miss a special beer I wanted or if I'm socialising with brewers or friends who I don't see much and I would rather be more sociable than writing notes. Also beer does change in a glass or in a pint or in batches. It changes when it leaves the brewery. The ingredients change all the time. This list will probably be different next year but it's great to see lots of innovative microbreweries doing well by producing styles which are not cheap to produce. The market has clearly changed which is great news for Elusive as the demand is definitely there for good and interesting beer. Gazza made an interesting point about having to withdraw a core beer because it didn't sell enough. His hop-forward IPA's and EPA's or APA's etc fly out! To brew more new different beers is definitely a challenge but it appears to be the way forward in order to satisfy the market.

    ReplyDelete
    Replies
    1. Thanks Tom. I do think having a diverse range helps keep things interesting for brewer and drinker but there's also a need to have steady sales, for example supplying the beer to accounts on a regular basis. The accounts local to me would probably struggle to sell hop-forward IPAs, so there needs to be something to accommodate them. Also, price is a factor of course.

      Delete
  7. Ask them their price point for buying and see if you can do some deals. I would do a standard porter or bitter for your way and keep it around 60 quid as an entry. Maybe a mild.

    ReplyDelete
    Replies
    1. It's about balance, really. Selling regularly at that price would see me going out of business pretty quickly.

      Delete
  8. It often seems that the most delicious flavour is "rare" on these rating sites. Maybe a few bottles of homebrew from a 20 litre batch would rocket to the top if sold in a trendy bar in London.
    Good graphing by the way. Always good to see a graph.

    ReplyDelete
    Replies
    1. Thanks Dave. Rarity may well be a factor, especially once there's a bit of momentum/reputation behind a brewery. Although, if I look at the top rated beers above, some of them are available fairly regularly and the one thing they all have in common is that they're excellent beers of course!

      Delete
  9. Thanks for the analysis - as a bit of a geek, I love it!!
    Worth noting another major difference between Ratebeer and Untappd - former is £2.99 on app store; latter is free... If you're trying to analyse the wider market, I know which I'd use...

    ReplyDelete