Sunday, February 27, 2011

All-in adjusted standard deviation, and what you should be using it for

The statistic frequently referred to as all-in adjusted winnings is designed to cancel out some "luck" effects in measuring winrates, and it differs from realized winnings in hands where all remaining players were all-in on the same pre-river street. For such hands, all-in adjusted winnings is defined as replacing realized winnings from the all-in pot with the the player's pot equity upon becoming all-in. In other words, this statistic gives winnings if every eligible pot were "run infinity times" (a la "run it twice").

Many are quick to point out that the all-in adjustment captures only a small portion of the "luck" in poker. It only captures "handwise" luck rather than "rangewise" luck, and only for the portion of times when players got all-in. Different play styles may lead to fewer all-in pots with certain types of hands. Nonetheless, it still always produces a more accurate measure of true underlying winrate.

All-in adjusted winrate are an unbiased estimator of true winrate

Though the intuition behind this is usually pretty clear, some people trick themselves into thinking that all-in adjusted winrate might be biased in one direction or another. I don't know how they manage to do so. It's easy to show that it's not.

Let X be the actual realized results in a given hand. Let Y be the results ignoring any money that came from a (main or side) pot that involved two or more players being all-in with cards still to come. Let Z be the results of such all-in pots. Then, clearly,
If we let A denote the all-in adjusted winnings from the hand, then, by definition,
and thus all-in adjusted winrate is an unbiased estimator of true winrate.

It's also intuitively clear and easily demonstrated that all-in adjusted winnings have lower variance than realized winnings. edit: It's actually not quite as simple as I first thought, since Y and Z are not independent. Since only one of Y and Z is positive in any given hand, we can condition on the event E that no all-in occurs, and let p be the probability of this event:
So all-in adjusted winnings are a better estimator for true winrate than realized winnings. Again, most people know this and do in fact use all-in adjusted winnings as an improvement over realized winnings in estimating their winrates.

All-in adjusted standard deviation

Here's something you probably don't know — Holdem Manager can calculate the standard deviation of all-in adjusted results. It's not included in the software, but it can be coded as a custom stat (credit due to nofolmholdm on the Holdem Manager forums).

Copy this text into a file called "customstats.txt" and put it into your Holdem Manager\Reports folder:

<Stat GroupName="Default" ColumnName="EVStdDevbigblind" ValueExpressions="10*stddev((case when EV.SklanskyBucks <> 0 then EV.SklanskyBucks else ph.NetAmountWon end)/GT.BigBlind) as EVStdDevbigblinds; 1 as DenomOfOne;" Evaluate="EVStdDevbigblinds/DenomOfOne" ColumnHeader="EV Std Dev\nbb" ColumnFormat="0.00" ColumnWidth="*" Tooltip="EV Standard Deviation" />

You can then add this to Holdem Manager reports like any other stat. You'll notice that the Holdem Manager summary output for the old data I used last time includes this.

The standard deviation of all-in adjusted winnings are 69-85% of the true winnings in this data, depending on game structure and how often one gets all-in. This is a bigger reduction than I expected! So, even though results of all-in pots is only a portion of the "luck" in poker results, it turns out to be a pretty significant portion. I'd say it's big enough that all of us should go through the trouble of using this statistic.

Be careful — all-in adjusted standard deviation does not reflect our true expected standard deviation like all-in adjusted winnings reflect our true expected winnings. All-in adjusted standard deviation reflects our true standard deviation in hypothetical poker games where we would always "run it infinity times". It does give us a glimpse of how much we can lower our "variance" by running it as many times as we can. But, until we can all start doing pure equity chops in the poker games we play, we don't get to use all-in adjusted standard deviation in any bankroll formulas.

However, in estimating our long-term winrates from observed data, all-in adjusted standard deviation will give us a better idea of how close our realized winrate is to our true winrate. When forming confidence intervals, if all-in adjusted standard deviation is, for example, 69% of the size of unadjusted standard deviation, the confidence interval it yields will be 69% of the size, which is a great narrowing of our interval for any given level of confidence.

For example, for the $1/$2 CAP data, if we ignored both all-in adjusted earnings (EV bb/100) and all-in adjusted standard deviation (Ev Std Dev bb) and simply used the unadjusted, realized earnings and standard deviation, assuming approximate normality and no Bayesian prior (I believe this is discussed in The Mathematics of Poker by Chen/Ankenman), our 95% confidence interval for our winrate in bb/100 is about [-1.77, 8.53].

If we replace realized winnings by all-in adjusted winnings, which improves accuracy, the 95% confidence interval is [-2.94, 7.36]... sadly, we had been running above all-in EV, so this interval paints a more accurate picture of what we should expect long-term.

If we then make the second change of replacing standard deviation with all-in adjusted standard deviation, we get to narrow the interval back down to [-1.51, 5.93]. This is our most accurate and most narrowed-down interval.


Put the all-in adjusted standard deviation stat into your Holdem Manager and use it, along with all-in adjusted winnings, when estimating your winrate. Easy game.

Caveat - card removal effects

In practice, database and analysis software such as Holdem Manager can only calculate the all-in adjustment assuming that all unrevealed cards are still in the deck. That is, in a situation where the cards that players have already folded are more or less likely to be of a certain rank or suit, since there is no way to know the identities of the folded cards, any practical all-in adjustment will be off by a little and will not use probabilities that accurately reflect the true pot equities.

It's never been established what the net effect of this is for different types of players. Since poker is zero-sum aside from rake, any all-in adjustment biases would cancel out entirely if the entire poker population were added together. Nonetheless, it is possible that a player with certain style of strategy amid certain game dynamics should expect a bias. Situations which bias it one way should be mostly cancelled out by situations which bias it the other way, though it's possible that the net effect would be nonzero.

Some forum discussions in the past have speculated that the net effect might cause all-in adjusted winnings to overestimate true winnings for winning players, but it is impossible to draw a firm conclusion from informal data sampling such as this, as those who "ran really bad" may be more likely to participate.

The bias it would introduce is certainly small, and we will ignore it. We will also assume that Holdem Manager and similar software are computing this accurately, though there have been issues in the past.

1 comment:

  1. You are an epic man sir

    I wish you win wsop main event one day


Related Posts Plugin for WordPress, Blogger...