Silver’s predictions for each state are made in terms of probabilities.  So, sometimes he’ll get those right and other he’ll get wrong totally based on random chance.  But how much should we expect from him?  Should we expect him to get every single one of his predictions correct?

Let’s assume that the probabilities he reports are 100% accurate (that’s a huge assumption, but I’m doing it) and we simulate the election thousands of times.  Based on these simulations, how many of Silver’s predictions do we expect him to get wrong.

Using Silver’s 56 state presidential win probabilities from last Friday, we can consider Silver’s prediction in each state to be the candidate who has the higher probability.  Now, taking those same probabilities, we can simulate an election based on those probabilities and compare those to Silver predictions.  Now, we can run this simulation thousands of times, and count how many times Silver’s predictions are expected to be wrong.  According to these simulations Silver is expected to get every one correct only about 5% of the time based on his probabilities if this election was run many, many times.  So it’s very likely that he gets at least one state wrong.  It’s to be expected.  At the other end of the spectrum, there is only about a 1.6% chance that Silver gets more than 5 states wrong.

This means we should expect that Silver , about 93.4% of the time, will get somewhere between 1 and 5 states incorrect.  55% of the simulations, more than half, have Silver getting either 2 or 3 states incorrect, and the average number of incorrect predictions in my simulations was 2.4522.

So, what should we expect from Silver?  We should expect him to miss at least one state, and most likely to miss 2 or 3 states.

Cheers.