So Lisa Goldberg gave a talk at Loyola Chicago on Monday afternoon about the hot hand in basketball where she presented a paper where she shows that there is no statistical evidence that the hot hand exists.  While we didn’t film her talk, this numerphile video is basically the same as what she presented on Monday:

The basic argument in the paper is that the probability of making the next shot given you made the previous shot is not statistically different than the probability of making the next shot given you missed the previous shot.  So I have a lot of thoughts on this, but first let’s talk about the really interesting history of this topic.

In 1985, what can very accurately be called the seminal paper from Gilovich, Vallone, and Tversky, which defined the hot hand and concluded there there was no statistical evidence for it.  For years this was considered orthodoxy in most of the sports statistics world, even though almost everyone in basketball feels that the effect is real.

Years after that paper was original published, in 2015, Jason Miller and Adam Sanjuro published a paper where they pointed out that the way that Gilovich et. al. (1985) went about looking for the hot hand was slightly flawed in that their were unaccounted for biases that are introduced in the streaks that were not accounted for when the streak length that is considered is small.

What they pointed out in this paper is really, really interesting.  So let me talk about it for a second.  Let’s say you have a finite string of coin flips from a fair coin.  Call 0 a tails an 1 a heads.   So you might have a string of flips like this: 001101001.  Now, for a fixed number of flips, what is the proportion of 1’s occurring after a 1 in a finite sequence?  It’s 50% right?  Right?!?!  It has to be 50%!

Turns out, it’s not 50%.  Andrew Gelman has an excellent explanation of the issue here.

Following the Miller and Sanjuro paper in 2017, Daks, Desai, and Goldberg published a paper where they updated Gilovich’s original paper using permutation testing to account for the bias that Miller and Sanjuro pointed out in their paper.  In Daks et. al.  they find that even when using the permutation tests and accounting for the bias, they still find no evidence of a hot hand.

This paper led to that numerphile video about the hot hand up at the beginning of this post, though Miller and Sanjuro don’t agree with the findings in that paper.

So what are my thoughts on this?  I don’t think that any of these paper are looking for the hot hand in the correct way.  I think you need to look at building some sort of mixture model or a hidden Markov model with two states representing hot and regular.  Once you fit that model you can look to see if there is a significant difference between the states and compare this model to a one state model and see which one gives you a better fit.  I’ve written about this type of thing before in baseball with Rob Arthur.

I also think, specifically in basketball, you absolutely cannot be viewing the data as a string of 0’s and 1’s.  If you only are looking at makes and misses you are ignoring so many other factors such as shot distance, game situation, distance to nearest defender, etc. that affect the probability of making a shot that need to be controlled for.  What’s nice about studying the hot hand idea in baseball for pitchers is that there are relatively few factors that need to be controlled for when looking at pitchers (runners on base, score, pitch type, etc.).  And it’s also easier to look at pitchers because there are no opposing players who are trying to hinder the pitchers ability to do what they are doing and the pitch is always coming from the same distance away (This is why I think bowling would be a nice place to look for the hot hand.  Someone get me the data!)  In basketball, everything is different from shot to shot.

So am I convinced that the hot hand exists?  My answer is really, truly, I don’t know.  I haven’t seen anything that convinces me it does or does not exists.  And also, it depends.  It depends on exactly how you define the hot hand.

Anyway……..

After Dr. Goldberg’s talk, I was lucky enough to get invited to dinner with her because, dammit, I’m important……….(Also, at the dinner were John and Sue Dewan and Lisa Goldberg’s Daughter)

While at dinner our department head introduced me as the director of our Data Science program and Dr. Goldberg asked me this following question: What are the three things you want students to take away from your program.

I stalled for a bit, and then just straight up said I’m going to avoid answering that, but I’ll tell you want to I think a Data Science student should know how to do.

Towards the end of dinner, I just had to ask Dr. Goldberg the same question she asked me.  What did she think the three most important things were (I hope i remember these at least somewhat accurately):

1.  Statistics does better with more data and but more data is harder for computers.  Dealing with this issue is fundamental to doing data science.
2. Remove your personal biases from the analysis.
3. Design your experiment before hand.

Pretty good answers.  But I’ve now been thinking about this question for a few days now.  So what are the three most important things that I want our Data Science students to know?

After thinking about this for a while here are my answers (in no particular order):

1. Always try to do the thing you are trying to do.  (For example, if you just want to build the best classifier model, you aren’t that interested in interpreting parameters.)
2. Data Science consists of two major parts: managing the data and analyzing the data.  Neither part is more or less important.
3. You can manipulate statistics to say many different things.  Be ethical.  (Present data to others the way you want others would present data to you.)

And of course, I want all of my students to know that the answer to virtually every single question in statistics is “It Depends”.

Ok.  Good night.

Cheers.