This morning Yahoo Sports published a piece titled “Op-ed: How one flawed study and irresponsible reporting launched a wave of CTE hysteria,” written by Merril Hoge, a former NFL running back and ESPN analyst, and Dr. Peter Cummings, an Assistant Professor with the Boston University School of Medicine. In it, they criticize the methods of an article that was published in the Journal of the American Medical Association (JAMA) in 2017 entitled “Clinicopathological Evaluation of Chronic Traumatic Encephalopathy in Players of American Football.” The JAMA article, as they point out, has been an important part of widespread discussions about football and brain injury.

Hoge and Cummings make three major points against the article (which they call strike 1, 2, and 3. Get it. It’s a sports reference!):

1. There was no control group
2. There was selection bias
3. Failed to control for external factors

And you know what? The authors are exactly correct.

What kind of moron would design a study without a control group? How can you design a randomized control trial (RCT) without a control group?!?! It’s right there in the name! It’s insane. Amateur hour. Next, as anyone with half a brain knows, you have to select your subjects randomly in a RCT. It’s also right in the name!! Randomized. Control. Trial. And then, get this, they didn’t even control for external factors! As Cummings and Hoge point out, “nearly half the players had a history of substance abuse, suicidal thinking or a family history of psychiatric problems.” Can you imagine not controlling for that stuff in a randomized control trial? How could anyone be so stupid?

Ok, seriously. How could someone do a randomized control trial without having a control group, randomization, and controlling for external factors?

Well…they didn’t. The JAMA study wasn’t a randomized control trial.  It was a case series.  The original authors made no attempt to do a randomized control trial. If you look at the original study, the authors note in the very first line of the findings section of the abstract that they are using a “convenience sample.” This means that they know they aren’t doing a randomized control trial. Which means they aren’t even attempting to show a causal link between playing football and CTE.  In fact, in the Wikipedia entry for case series it explicitly mentions that “unlike studies that employ an analytic design (e.g. randomized control trials)…case series do not…look for evidence of cause and effect.” This is how the actual authors summarize their findings in the conclusions section of the paper:

In a convenience sample of deceased players of American football, a high proportion showed pathological evidence of CTE, suggesting that CTE may be related to prior participation in football.

They NEVER claim a causal link. Which makes this statement by Hoge and Cummings all the more odd: 

Then we took a closer look at the study that led to the Times story — apparently something few journalists had bothered to do. When we dug into the methodology, we were floored. The study was so badly flawed that it was nearly worthless. But that’s not what had been reported in practically every major media outlet in the world. Thanks to the barrage of sensationalist coverage, the “110 out of 111 brains” story had turned into a wildfire, and we were standing around with a couple of garden hoses, telling everybody to calm down.

They criticize the authors of the Times story for not taking a closer look at the original paper. But I wonder if Hoge and Cummings actually read the original paper. Their article on Yahoo Sports criticizes three aspects of the original study that literally aren’t a part of the original study.

The cynic in me thinks maybe all of this is just a ploy to sell more books. Because as you’ll notice, Hoge and Cummings are promoting a book called “Brainwashed: The Bad Science Behind CTE and the Plot to Destroy Football”, which I’m not going to link to because, based on this Yahoo Sports article, I’m guessing it’s trash*.  

Side Note 1

I will admit that Hoge and Cummings do have a point that the media coverage of this research was fairly skewed.  The media plays up the potential link between football and CTE and maybe doesn’t fairly state that as of yet no causal relationship has been established between playing football and CTE.  But even if the media coverage was fair, what Hoge and Cummings should be doing is calling for more, high quality research into the relationship between football and CTE instead of trashing a case series for not being a randomized control trial.  Because right now the truth is that we simply don’t know if there is causal relationship.  But we also don’t know that there isn’t a causal relationship.

Side Note 2

Smoking and lung cancer is a famous example where there was clearly a correlation known for a long time, and for decades people were talking about how the relationship wasn’t shown to be causal. Eventually, scientists were able to show a causal link. But it was all the demonstrated correlation between lung cancer and smoking that led people to study the causal relationship between the two. So, this CTE study is important not because it shows a causal link—which, again, no one is trying to demonstrate here—but because it is consistent with what we would expect if the relationship between CTE and football were causal, and will lead to more work on studying that relationship. This is an important part of how science works.

And finally, remember, “correlation does not even imply correlation”

*I need to be clear that I HAVE NOT read the book.  The book could be amazing.  I’m ONLY commenting on the Yahoo Sports Op-Ed (added 10/24/2018 – 8:03am).