What Monte Carlo or Bust (and the whole gambling industry) gets wrong

I've recently been reading a new book out by Joseph Buchdahl entitled "Monte Carlo or Bust: Simple Simulations for Aspiring Sports Bettors". I'm a fan of Joseph's work more generally and am naturally drawn to anything that tries to analyse sports betting from a principled analytical perspective and the book delivers on many fronts in this regard. 

However, I believe the book makes one key mistake in it's choice of the main metric used in many analyses which affects a number of the chapters: the analysis and conclusions based off this metric end up reflecting the fundamental error of the metric itself and a number of these conclusions end up looking meaningless once you understand what's driving them.

I don't blame Joseph for this choice of metric - it's the de facto standard the whole gambling industry uses for the same problem - it's only when someone tries to run with it as far as Joseph does in his book that the flaws become so apparent. In this post I'll try and explain what I see as the flaw behind the metric and how we can remedy it in a more principled way.

The Problem

Sports bettors and Bookmakers alike want to answer the same kinds of questions about their trading - how profitable are they? What margin do they make, or expect to make? How should they compare different trading performances or strategies?

The starting point for coming up with a metric to do this kind of analysis always has to begin with the thing everyone is trying to maximise - the profit. As the profit of any gambling strategy is itself a random variable an alternative analysis will use the expected profit instead, conditional on some kind of probability model of the underlying events - normally referred to as the 'expected value' or EV. Either way, essentially everyone will agree on profit (or it's expectation) as being a core component of any analysis metric.

However, comparing profit alone between strategies isn't really an apples-to-apples comparison. It's fairly intuitive that if I make a £10 profit by placing a thousand 10p bets that's very different to if I ended up at the exact same profit after a rollercoaster of placing a thousand £100 bets; profit or EV alone as a metric is missing something critical about the 'amount' of betting that was done to achieve it.

What we really want is to decide on some way to put the profit generated into perspective so we create a metric which is more comparable on a like-for-like basis regardless of how much betting was done. If Profit or EV is the numerator of our fraction, then what is our denominator?

The Mistake

This is the point I think the gambling industry (and, unfortunately, also Joseph's book) makes a crucial, although very understandable, mistake. It's a very natural mistake, because so much of sports betting is fundamentally around binary outcomes, where one risks X to win Y. In gambling parlance, the amount you risk is the amount 'staked', so the sum of the stakes seems like a very natural denominator to use in our fraction. The whole interface of the industry is built around you keying in a number into a betslip, so surely that number is a good choice of the 'amount' of betting that's going on?

Unfortunately I believe this choice of denominator is a poor one and leads to a number of erroneous conclusions if taken as axiomatic. Let's try and work up from first principles what would make a good denominator for analysing sports bets by going more general and thinking about what metrics would work for any trading strategy. Once we relax our assumptions around binary options (essentially what a normal sports bet is) it becomes more obvious what the correct approach is by leveraging the financial literature on the same topic.

The general approach from finance is to work out your returns (or profits, in the gambling context) adjusted for risk. Returns alone are meaningless because they don't reflect how much risk you had to take to achieve them; once you correct for this you get something that allows you to compare very different strategies by a single metric. Probably the most famous financial metric which does this is the Sharpe ratio, which is the ratio of the average return to the standard deviation of those returns. Indeed it is so famous it gets at least one mention in Monte Carlo or Bust, although the connection between "returns per unit stakes" and "returns per unit standard deviation" isn't noted - the two are used in different contexts.

Why it's a mistake

One the general concept of "returns per unit risk" is made more explicit, we can then evaluate different definitions of 'risk' and see whether they seem to have the properties we expect of good risk metrics. If we consider stakes as just one possible risk metric, we can see it doesn't behave according to an intuitive understanding of what risk means via a simple thought experiment. Suppose I told you I have wagered all of your net assets on a single outcome, either a) whether San Marino will win the next World Cup, at odds of 10,000 or b) whether the Sun will rise tomorrow at odds of 1.0001. In both cases I have staked the same amount, however in one case you will sleep soundly (and thank me for the free money) and in the other you will despair at my actions, having almost certainly lost your entire net worth. Clearly the 'risk' of these two actions is not identical - stakes systematically overweights the risk from very low odds and underweights the risk from very high odds.

This bias in risk mis-attribution essentially explains all the odds-related phenomena discussed in the book when analysing returns per unit stake as the yardstick by which to measure everything by. Multiple chapters end up discussing variations of how the expected value yield (a term for returns per unit stake) of different profitable strategies is 'greater' at high odds, how it's much easier to get a good margin at higher odds etc. Once you're aware of this issue, it becomes hard to work out which conclusions are unaffected if the analysis had been done using a better risk metric and which ones are just artifacts caused by the bias inherent in returns per unit stake.

This is essentially the same mistake the bookmaker industry makes when claims such as 'Multiples are great because they are really high margin' are thrown around (as it commonly is). Multiples, if not short-paid, are essentially the same margin as their Single bet constituents which makes them just as profitable (or unprofitable, against a sharp client) as those Singles are. Another good book, the Logic of Sports Betting, has a chapter "Taking Advantage of Parlays" that does a good job of arguing the same case here, although it takes the perspective of comparing Multiples vs chaining many Single bets together separately (which is, indeed, exactly what a Multiple is!) and ends up committing the same industry-wide sin of believing amount staked is a good unit of account.

Once you've realised how returns per unit stake is just a specific case of the more general returns per unit risk, it becomes obvious how to analyse some of the more 'exotic' bets where the payoffs are no longer binary and the 'stake' seems less intuitive as a risk measure (if it even exists at all), e.g. spread bets and Asian handicap bets on quarter or zero lines. In all cases the exact same general approach can be taken and very different looking bets can be compared side-by-side.

Asia gets it right

The only part of the industry I believe has cottoned on to the flaw in stakes as a measure of risk or betting volume are the Asian bookmakers and the credit market. For this, I think the 'skin in the game' effect is the driver behind Asia being one step ahead of the rest of the world here, and that's because the Asian credit market needs to pay rebate.

For those who aren't too aware of how the credit market works, essentially Bookmakers pay 'Agents' to bring them clients and remunerate them for doing so. Some Agents will even pass on some of this cut from the Bookmaker on to the end client as a reward for sticking with the Agent & Bookmaker. This raises the question of what metric to remunerate the Agents on. Rather than use stakes, the Asian Bookmakers have chosen to go for the absolute value of the profit or loss for each bet, colloquially it's "the money that changes hands" e.g. if one bet wins £10 and another loses £20 then rebate is paid as a percentage of £30 (there is another way Agents can profit via something called position taking, but that's worth a separate post in it's own right). I'll refer to this as the Asian turnover metric.

As a risk metric, this is a much better one than amount staked. Indeed it's another common risk metric from finance/statistics known as the mean absolute deviation. It's the non-squared version of the standard deviation, and indeed there's some degree of statistical debate as to whether it's preferable to standard deviation as a unit of risk (check out this excellent paper cited in the StackOverflow answer if you are curious to learn more). The reason I believe Asia chose the absolute profit and loss metric over stakes is that the latter is easily gameable by a motivated gambler (of which there are plenty!) and the former is not.

To give an example of how you'd game a stakes-based rebate, consider a very standard bet in the Asian market - the Asian Handicap bet. I'll assume familiarity with the mechanics of AH betting. Consider a bet on the +0.00 line of a football match in the 95th minute of play. At this point, the bet is very very likely to be 'pushed', i.e. no goals are scored and the handicap bet pays out £0. If rebate was paid on stakes one could bet very big and gain a lot of expected value just from the rebate alone, as even when the bet is pushed you'd be paid rebate. If this was done systematically by sharp punters to gain their share of the rebate, it would get very expensive very quickly for the bookmaker. Absolute profit or loss, being a good risk metric, cannot be gamed in this way; it only rewards clients who take genuine risk in their bets.

Rant over

I hope this post can be seen in the manner it is intended - as constructive criticism of the use of stakes as a unit of risk - and not as an attack on Monte Carlo or Bust more generally. I do believe the average sports betting enthusiast would still learn a great deal from Joseph's book; even as a veteran I still find seeing simulations of betting returns very sobering when you know the amazing consistent winning streak is generated entirely from a negative EV strategy and a reminder that only statistical analysis can tell you the truth of whether something is signal or noise (or at least tell you you don't have the data size to know!). The chapter on closing line value is, I think, the best discussion of the topic I have ever read.

I was motivated to write this post in part out of frustration from not living in the parallel universe where the betting industry uses the Asian turnover metric as standard and all 'margins' and 'yields' implicitly use it; that universe's version of Monte Carlo or Bust would be an excellent read indeed.