A few months ago, my friend & writer Noah Davis asked me a question that was bothering him. I’ll paraphrase, but this was roughly what he said:

Does consistency matter for quarterbacks? Like would you rather have an average QB who is never really great, or a good QB who occasionally sucks?

Well, fortunately there are ways to measure performance consistency, and one of them is standard deviation. QB’s with high standard deviations in their game-by-game metrics are the less consistent ones, and visa versa.

But perhaps an even better idea than just measuring each QB’s standard deviation of a certain metric is to compare the overall distribution of performance. This can be done using many tools, and we chose density curves, which are just rough approximations of the smoothed lines that one would fit over a histogram.

The culmination of our project into looking at QB density curves is summarized here on FiveThirtyEight. In addition, I…