Thursday, 5 June 2014

Lucky Is As Lucky Does - A Study In Variance

It's been a long while since my final Netrunner tournament report got cut off midway through by the onset of extreme apathy.  Apologies to those still waiting on tenterhooks to learn about how that random tournament went (short story: I won some games of Netrunner and lost some other games) but since then I haven't lifted an Icebreaker or Asset in anger, and to be honest I haven't really felt like I missed a thing.

However something happened this past weekend that has given me something that I want to write about, and a lesson that I think it's important to try and teach.  This is going to be a blog inspired by my experiences at the Magic: The Gathering Grand Prix in Manchester, but fear not Netrunners because I think the lessons apply to most card games, Netrunner included.  I'll try to keep it grounded enough in clicks and credits that you can follow me.

I want to talk about a vitally important (but often-ignored) topic: I want to talk about Variance.

Variance 101

Variance is a term that that comes originally from statistic and probability analysis, and it has bled into CCGs/LCGs primarily via the players who crossover into Poker tournaments.  In a nutshell Variance is used to define how much each individual result can be expected to vary away from the average result. For example, consider if I asked a hundred people to pick a number between 0-20, the average would be 10.  If I then asked a hundred people to pick a number between 8-12 the average would remain 10, but the answers would be much more tightly grouped.  
In a 0-20 range there is a higher 'variance' on the possible answers than in an 8-12 range.

Variance is closely related to what you may classically have thought of as being 'lucky' or 'unlucky'.  In truth the terms are pretty much interchangeable but because luck tends to bring a lot of superstitious connotations along with it most serious players prefer to talk about variance.  Once you stop talking about luck and start talking about and understanding variance you can start to take control of that element of the game you are playing and you can make it work for you.

Let's talk about what happened to me in the Magic: The Gathering Grand Prix in Manchester.

Firstly: I don't play Magic: The Gathering.  

Don't get me wrong, I used to play Magic A LOT.  Back in the late 1990's I was probably playing Magic up to a dozen hours a day and when I wasn't playing Magic I was thinking about playing Magic.  It was something of an obsession.  I was playing in tournaments every week, I was travelling around the UK to PTQs and (once or twice) earning my stripes on the Pro Tour - I wasn't great at the game but I was definitely very good at it (probably in the top 1% of players).  In the 2000's I walked away from Magic before returning between 2005-13 as a coverage writer, working for Wizards of the Coast at Grand Prix and Pro Tours by providing match reports, players interviews and strategy insight.  

So it's not like I was completely out of touch with the game, but I had pretty much stopped playing entirely.  I've worked out that before Grand Prix Manchester my last tournament had been the Gatecrash pre-release nearly 18 months earlier, and before that it was Grand Prix Toronto at the back end of 2010.

On top of being out of the game a long time my preparation for Grand Prix Manchester was precisely zero.  On the Friday I phoned a friend who was at the venue and asked him to register me into the tournament, and he agreed to lend me a deck to play.  On Saturday morning I collected the 75 cards I was going to play with and sat down to play Round 1.  I wasn't there to win, I was there to hang out with my Magic friends. 

The format was Theros Block Constructed and I hadn't even seen a single Theros card.  I didn't know what any of the cards in my deck did.  I didn't know what any of the cards in anybody else's deck did.  I didn't know any of the decks in the format and I didn't have any byes to help me out.  

In short: I didn't know jack and I was starting from the basement against players who knew more, played more, and in many cases had the head start of byes.  I was dead meat and I knew it.  I figured if I did really well I might be able to win more games than I lost and finish on a 5-4 record after the first day, but I knew even that was a serious long shot.  

There was a realistic chance I would finish dead last, but out of over 1,400 players I finished 81st and won $250, with a final record of 10-1-4.

Why did I do so well?  Because I embraced variance.

Variance in Deckbuilding

Not all decks are made equal.  We commonly talk about decks being 'better' or 'worse' than other decks and while it's certainly true that good and bad decks exist there is also another subtle factor that is often overlooked - how much variance the deck allows to occur.  

Some decks are capable of getting a particular opening hand or combination of cards that will mean it beats almost anything that the opponent can do - it's just an unstoppably powerful opening that either wins outright or puts you so far ahead that it's an unassailable lead.  The price for having this potential for extreme success is usually including the potential that whenever your opening hand is poor, or you don't see that particular combination, you're at a significant disadvantage.  These types of decks are high variance strategies - they can be extremely powerful in one game, then completely impotent in another.  Other decks, however, can build in layers of redundancy and support so that they're very consistent in executing their strategy, although the price for that reliability is usually that they sacrificed the potential for that unstoppable knockout blow.  High variance is 'go big or go home', while low variance is 'we'll be there or thereabouts'.

A classic Netrunner example would be the difference between the Criminal IDs: Gabriel Santiago and Andromeda.  Andromeda's huge opening hand size means that she is very consistent at seeing the cards she wants to see - she's the poster child for a low variance strategy.  Gabe starts the game without Andromeda's 80% opening hand size boost but what Gabe brings to the table is a powerful ability to gain a lot of credits freely and easily once he's set up.  

Compared to Andromeda's reliable and steady approach, Gabe is high variance - he's far more likely to draw an opening hand that doesn't hold Desperado/Account Siphon/Icebreakers, but if he DOES draw those cards then he is virtually always more powerful than Andromeda thanks to the free credits he gains from accessing HQ.  

Variance can also be a factor of how much the deck wins/loses depending on the deck it plays against, for example Cerebral Imaging is quite a high variance strategy in that it is very strong against most Runners but very weak against Gabe's Sneakdoor Beta and HQ accessing plans.  Individual card choices can also make significant impacts on the variance of a given deck - choosing to take out three copies of Plascrete Carapace to play Same Old Thing is a double-edged sword as you've made your deck much worse in matchups against Scorched Earth but better in matchups against pretty much any other Corp deck... you've increased the variance of your deck.  Another classic example is Anonymous Tip vs Jackson Howard - Jackson Howard is steady reliability while Anonymous Tip is short term power traded for long term consistency.  Maker's Eye vs R&D Interface is similar from the runner's point of view.

Variance is therefore a characteristic of the deck you choose to play and it's in your power to determine how much variance you allow into your strategy.  Hopefully you see now how we can begin to take control of variance and use it to our advantage.  We've taken a series of steps away from the idea of "I am lucky" being a characteristic attached to you that was out of your control, and how those steps have brought us to "I have chosen to use a high variance strategy" - a characteristic attached to the decisions you've made in deck construction/selection.

Variance is another tool in your deckbuilding arsenal.  Learn when to maximise and minimise its influence and it will serve you well.

Making Variance Your Friend

I talked earlier about an example of asking people to choose a number between 0-20, well lets turn that into a game.  We're going to use dice to choose a number between 1-20 with the aim being to roll the highest number, but you've got a choice about which dice you want to use:

  • A single 20-sided dice (1x D20)
  • Two 10-sided dice (2x D10)
  • Three 6-sided dice (3x D6)

Lets compare these options:

What sounded like a pretty simply decision is in fact quite complex.  The D20 is a high variance strategy - you've got the highest chance of rolling a number of 16 or more (25%) but also the highest chance of rolling 5 or less (also 25%).  At the other end of the scale is 3xD6, which is the low variance strategy - it's impossible to roll a 1 or a 2 as your worst possible result is to roll three 1s, but it's also impossible to score 19 or 20.  With 3D6 the odds of rolling particularly low or high scores is much lower than it is with a D20 (just 4.6%) and in fact the variance is so low that you've got a 48% chance of rolling a pretty middling score between 9-12!  Sitting in the middle of the variance range is the option to roll 2xD10, and in fact this option has a little unfair advantage over the others - it's impossible to score just 1, but still possible that you'll score 20 by rolling two 10s - the Mean score is therefore slightly higher and the odds of scoring a high number are better than your odds of scoring a low number.

So, in the unlikely event that somebody ever offers to play this game with you, statistically you should play with 2xD10.

But that's not all, in fact I'm just getting warmed up.  

The above is true if the game only involves two players, but what if there's more players?  What if there were 8 players all rolling dice with the prize going to the highest number?  If that's the case then you can be pretty sure that at least one of your seven competitors will roll a high number, so in order to win you'll need to roll at least a 16 to have any chance of winning, and ideally something like an 18 or 19.  Suddenly that D20 option starts to look pretty good - it gives you a 25% chance of rolling 16 or more, and you've got a 5% chance of rolling a straight 20, while the 2xD10 only has a 1% chance of scoring 20 and the 3xD6 can't score it at all!

So if you're playing the game with 8 players, you probably want to use the D20.

But that's not all, let's take it even further!  What if we change the prize structure of the game?  What if instead of giving out prizes just to the winner we just give out prizes to the Top 6 players, with the Top 2 players getting a a bigger prize than the others?  Now you've got a really tough decision to make.  The best way of ensuring you don't roll one of the lowest two numbers and miss out on a prize is by rolling 3xD6, but if you roll 3xD6 you're probably ruling out your chances of finishing in the Top 2 and getting the biggest prize.  The D20 becomes a huge wild card because you could get nothing or everything, while the 2xD10 plan sits back into the middle of the pack - you've got some chance of coming in the Top 2 and a reduced chance of missing out entirely.  What it comes down to now is a personal choice - what's important to you?  Do you want to win the big prize or do you just want to avoid coming away empty handed?

I could expand this example further and further if I cared to.  What if the prize structure was weighted with big prizes at the top but very little at the bottom?  What if the difference between 6th and 1st was pretty small but coming 7th meant missing out on a lot?  Beyond that there's a metagame - what if you think a bunch of people are going to choose to roll D20, how does that change your decision?  What if you think most people will roll 3xD6?  What if there was the option to roll 5xD4?

We'll return to the dice game shortly, for now let's talk about how I decided what deck to play in the Grand Prix.

Variance, Skill and Heroics

At Grand Prix Manchester there was a choice of three decks available to me and the one that I ended up playing was the one that seemed like the strategy with the highest variance: U/W Heroic.  

This was entirely deliberate.  My one real target at Grand Prix Manchester was to win enough games on Saturday to make the cut into the second day, with the chance to win up to $4,000 for the ultimate winner of the tournament on Sunday.  To qualify for Sunday I would have to win AT LEAST seven of the nine rounds on Saturday, each of which would be played against players who knew the format far better than I did.  To use the dice example: I needed to roll a high number if I was going to progress to play on Sunday because there were another 1,400 players all trying to roll a big score as well and I needed to be in the top 10% of scores.

As I've mentioned, there were three decks that my friends had available to choose from, each of us would play one of these decks:

WBG 'Junk'

'Junk' is perhaps the defining deck in Theros Block Constructed, bringing together most of the best cards in the format into a single deck (Sylvan Caryatid, Courser of Kruphix, Fleecemane Lion, Elspeth, Hero's Downfall).  Undoubtedly 'Junk' is a very powerful deck but it was also going to be one of the most commonly played and you could expect to face many matches against virtually identical decks.  'Junk' was the low variance option - very solid and very reliable, but with little ability to create rapid blowout wins.  In dice terms, it's 3xD6.

Mono-Blue 'Scourge'

The mono-blue control deck my friends had built was a good metagame contender, in that it played cards that weren't naturally amazing but which were well-positioned against the most popular decks in the format (Whelming Wave, Scourge of Fleets, Perplexing Chimera, Prognostic Sphinx).  Scourge was the 2xD10 option - it was quite consistent because there was a lot of redundancy in the cards it played but there were some matchups that were distinctly better than others, so there could be a higher range of variance due to your matchups.

U/W Heroic

The high-variance option, U/W Heroic pretty much always wins or loses the game in the first few turns.  You need to play a creature on your first or second turns, and then you need to rapidly make it both bigger and unblockable by layering Aura cards onto it, each Aura triggering the creature's 'Heroic' keyword to make the creature bigger (Netrunner parlance - you're playing Darwin and 16 copies of The Personal Touch). By the fourth turn you either have a huge unblockable creature that the opponent couldn't deal with, or you've probably lost.  U/W Heroic could beat any deck with the right draw, but a bad draw could easily leave you helpless.  That's pretty much the definition of a high variance strategy... 

There's was another factor behind my decision to play the high variance strategy, though, and it's a critical one.  Rather than just needing to score in the top 10% of players I knew that I was going to be one of the worst Magic players in the Grand Prix.  Not because I'm completely rubbish at the game but because I was completely out of practice and completely out of touch with any of the cards or decks in the format, while most other players had been practicing for weeks.  I needed enough positive variance to not only score within the top 10% of players in the room, but enough positive variance to also wipe out my natural disadvantages in this particular format.  

You can feed the impact of player skill into the dice game to illustrate how much this changes your decision making.  Let's say you can apply a modifier to your roll of between -5 and +5: a -5 is somebody playing the game for the first time, while a +5 is the best player in the world.  Let's repeat that game of the 8 players trying to roll the highest number, but now let's see how the decision changes for a player with -2 modifier, and a player with a +4 modifier (for the purposes of the example we'll assume everybody else in the game has a 0 modifier).

A -2 modifier probably accurately reflects where I was at going into Grand Prix Manchester - if it had been 1998 and I was playing regularly and on top of the format I might be worth as much as a +3, but my complete ignorance of the format was easily knocking 5 points off that, leaving me at -2.  From that position the odds are stacked against me whichever dice I choose to use, but although the D20 gives me the highest odds of rolling a low number and crashing out of the tournament straight away it's also the one that gives me my best chance of scoring something that might be high enough to make the cut. 

A +4 modifier probably represents the very best players who were at Grand Prix Manchester - your Kai Budde's, Samuele Estratti's and Martin Juza's of this world.  To the best players a high variance strategy like rolling the D20 represents an unnecessary risk - a 30% chance that you'll still wind up in the bottom half of the results.  Rolling 2xD10 instead gives you the same chance of rolling a high number and doing well and halves your risk of a low number (15% vs 30%).  You can also go one further and cut those odds again by rolling 3xD6, although this time you also cut your chances of rolling a really high number.

In a Magic: The Gathering Grand Prix many of the top players will choose to roll 3xD6 every time.  They know that in an open field of 2,000 players their chances of actually winning the top prize are slim no matter what deck they play, but they want to ensure that they finish in/around the Top 100 where they can pick up some money and valuable Pro Points.  It creates a pattern repeated again and again at Grand Prix around the world - the Top 8 and winner are rarely one of the very best players in the world, but if you check out who came in the Top 64 you'll find the same names popping up time and time again, quietly putting away cash and Pro Points outside the Top 8 limelight.  Grand Prix are such big and competitive events that it's impossible to guarantee finishing in the money every time, but by playing a low variance strategy the best players maximise their chances of consistently doing so thanks to their skill advantage.

Summing Up

Replacing 'luck' with 'variance' brings it under your control.  When you're building and selecting your deck you are able to make decisions that will either increase or decrease the amount of variance you will experience when you play the deck.  Increase variance and you buy a shot at getting lucky, decrease variance and you limit the risk of being unlucky.

What are the factors you need to take into account?

How many people do you need to beat?
The more people you're up against the harder it is to finish ahead of all of them, and the more positive variance you will need in order to be the champion.

How are the prizes distributed?
This can make a huge difference to your decision.  Two Netrunner tournaments could have exactly the same players in but require completely different approaches.  Consider how you would approach a tournament where the thing you really wanted was a Top 8 playmat and alt-art card, then consider how you would approach playing in Nationals where the prize for first place is a flight and invite to Worlds while second place only gets a playmat.  With so much of the prize pool focused onto first place a higher variance strategy becomes more attractive.

What does success look like to you?
This is personal choice - you may enter the same National Championships and say "Realistically I just want to finish in the Top 4, if I win that's amazing but just being in the Top 4 would be great".  That requires a different approach than that of a player who says "I want to go to Worlds - second place is as good as last" - the second player should embrace variance and try their luck, win or bust.

How good are you, relative to the other players?
This can require some real soul-searching, but if you decide that you're either significantly better or significantly worse than the people you're up against it can be a powerful factor.  It helps in deciding whether you attempt to limit the impact of variance and simply win games by being better at the game, or whether you throw caution to wind and let the fates decide if you can perform way above your skill level should allow.

At Grand Prix Manchester I was dead meat going into the tournament.  I didn't know what any of my own cards did, let alone the cards of anybody else.  But despite all this I finished ahead of 1,320 other players, coming 81st and winning $250.

Did I get 'lucky'?  Yes, of course I did (as well as playing pretty well, IMHO, given my ring rust).  But I had made a decision to allow myself to be lucky.  I had weighed up my chances and decided that my best chance of success was to play a high variance strategy.

Did I get 'lucky'?  Yes, but I was also smart.

The classic line from Dirty Harry is "do you feel lucky, punk?".  

When it comes to working out how you should approach variance in Netrunner (or any game) the real question should be: "do you need to to be lucky?"

Well.  Do you?