Battleship calculator

Originally posted on God plays dice:

C. Liam Brown has built a Battleship probability calculator, which (roughly speaking) works by finding the square which is the most likely to yield a hit given the set of hits and misses so far. You can play against it if you want. A lot of this might be said to be a web-friendly implementation Nick Berry’s analysis of the game, although analysis and implementation are two different beasts. (Funny, that keeps coming up in my day job…)

View original

An example of a statistically misleading MMR vaccine argument

I woke up this morning to a twitter comment about the “CDC Whistleblower Saga” from last year from one of my favorite twitter followers.  This obviously led to a conversion explaining to me that: vaccines aren’t effective, the idea of herd immunity has been debunked, they are making kids sick, and they cause autism (Vaccines Don’t Cause Autism). I should note that none of these claims have any scientific backing to them.  Other twitterers also told me that vaccines aren’t 100% effective (true; MMR is about 93% and 97% effective for 1 and 2 doses, respectively) and vaccines have side effects (also true, though side effects are rare).   But also, not a logical argument against vaccination.  I think we often forget (or in my case, never saw (thank you vaccines!)) how bad the measles really is (the measles are horrible).

And I know deep down, that no matter what I say, I’m not going to change someone’s mind on twitter.  So why do I engage in “discussions” with people like this.  I guess first, I can’t help myself.  If someone engages me first, and they are wrong, I’m going to tell them that they are wrong.  Though I’m not sure this is the best way to effectively deal with the anti-vaccine crowd (Here is how Jamelle Bouie of Slate suggests dealing with them), but I can’t help myself.  I do try not to insult or attack people, but rather their arguments.  But I find this difficult to do sometimes when I believe that these people are actively causing harm by trying to spread their anti-vaccines beliefs.  (So if I insulted you today, I apologize to you.  I should be better than that.  But I still think your ideas are pseudo-scientific cray-ball wackadoo stuff).

But, secondly, I am absolutely fascinated that people think this way.  It’s so foreign to how I think about the world.  I know people who are espousing these beliefs actually believe them in spite of the mountains of evidence against their claims.  To this end, The Atlantic wrote a really interesting article last fall about the psychology of anti-vaxers.  It’s a fascinating read.   And a bit sad with quotes like this: 

Dr. Douglas Hulstedt, a pediatrician in Monetery, California, shares Smoot’s preference for personal stories over scientific evidence. Hulstedt accepts patients who are not vaccinated. He goes even further, and recommends refusing vaccinations if a patient has a family history of autism, lupus, Crohn’s disease, or Type 1 diabetes.

“Why do I need a medical study?” he says. “If 80 percent of the parents of children with regressive autism in my practice say their child reacted after the MMR [measles, mumps, and rubella] shot, why do I need a medical study?” Hulstedt says that studies showing no link between the MMR vaccine and autism or showing that vaccines are safe and effective might have “fraud in the reportage.”

This is a medical doctor posing the question: “Why do I need a medical study?”.  That is absolutely appalling and evidence why I believe medical doctors need more statistical training before, during, and after medical school.  Statistics is a complicated subject.  Statistics is hard.  I find it is constantly difficult, and I’m supposed to be the “expert”.  But it’s just a difficult subject to tackle.  Statistics is hard.   But we need it as part of the scientific method to objectively answer medically important questions.  Like do vaccine work (Yes).

But it’s so easy to make mistakes.  As an illustration, let’s consider the plot below which shows measles deaths per 100,000 people over time.  This was sent to me by my favorite twitter follower with the (sarcastic) text:

As you can see #MeaselsVaccine [sic] is instrumental in eradicating #Measels [sic]

Screen Shot 2015-07-22 at 3.51.50 PM

They are arguing (I believe) that the death rate from measles was dropping for decades prior to the introduction of the measles vaccine, and the measles vaccine did little to lower the death rate of  measles.  So it follows that the measles vaccine isn’t as effective as science makes it out to be, therefore CONSPIRACY! #tinfoilhat #jadehelm

In all seriousness though, if you have no statistical knowledge, this might seem like a convincing argument.  And I’m sure there are a lot of smart people (and not so smart people) who could be convinced by this plot.  The problem with this is that this “analysis” is inherently trying to isolate the effect of vaccines on death rates without controlling for any other factors that are related to the death rate.  Medicine advanced quite a bit from 1840 to 1940 and the probability of dying from measles dropped considerably.  Even with no vaccines.  But that’s all this plot is demonstrating.  And it’s offering almost no evidence as to the effectiveness of the vaccine and is a case study in confounding.

I’d also argue the that graph is potentially misleading the viewer with scales.  By the time the vaccine is introduced in that graph, the line is so close to 0/100,000 that it’s hard to see the relative effect of the vaccine.  The death rate could have dropped 10% or 90% (It does drop some amount) and the viewer wouldn’t be able to tell .  The graph would be much stronger if it was zoomed in on the years 1948 to 1978.  But that doesn’t seem to be the narrative that is trying to be passed on with that graphic.

Screen Shot 2015-07-22 at 3.54.16 PM

To really get a handle on the effectiveness of vaccines, we should be looking at cases of measles rather than the measles death rate.  The graph below shows cases of measles in the US from 1954 through 2008.  The first vaccine was introduced in 1963 and a second version was released in 1968.  Notice the large and immediate drop from 1963 to 1969.  It’s possible that there could be some huge confounding effect that explains this drop, but I think it would be difficult to present a reasonable confounding effect here that would dwarf the effect of the introduction of vaccines.  The decline in measles cases was immediate and rapid.  So are vaccines effective in reducing disease?  Yes. Yes. Yes. Yes. Yes. and finally Yes.

Finally, I’ll close with this advice from the World Health Organization (WHO) an trying to persuade the anti-vaxxers:

How one addresses the anti-vaccine movement has been a problem since the time of Jenner. The best way in the long term is to refute wrong allegations at the earliest opportunity by providing scientifically valid data. This is easier said than done, because the adversary in this game plays according to rules that are not generally those of science.  This issue will not be further addressed in this paper, which aims to show how vaccines are valuable to both individuals and societies, to present validated facts, and to help redress adverse perceptions. Without doubt, vaccines are among the most efficient tools for promoting individual and public health and deserve better press.8

You can (and should) read the whole paper here.


Will 2015 be the Beginning of the End for SAS and SPSS?

Originally posted on

[Since this was originally published in 2013, I’ve collected new data that renders this article obsolete. You can always see the most recent data here. -Bob Muenchen]

Learning to use a data analysis tool well takes significant effort, so people tend to continue using the tool they learned in college for much of their careers. As a result, the software used by professors and their students is likely to predict what the next generation of analysts will use for years to come. I track this trend, and many others, in my article The Popularity of Data Analysis Software. In the latest update (4/13/2012) I forecast that, if current trends continued, the use of the R software would exceed that of SAS for scholarly applications in 2015. That was based on the data shown in Figure 7a, which I repeat here:

Let’s take a more detailed look at what…

View original 1,093 more words

Golfing with Dad

For the last nine years I’ve gone on a golfing trip to Vermont with my dad.  After the first year we started a competition complete with plaque and names engraved for the champions.  Here are the rules.  We play 7 rounds over the course of 4 days and you take your lowest score on each hole from any of the rounds and fill out one score card.  Whoever has the lowest “master card” wins.  (Ties are broken by looking at who had the lower score on the highest handicapped hole, second highest handicapped hole, etc.)

Currently, the series is 5-2 in favor of the dad.  Here is a plot of what this competition looks like when I’m playing well.  I’m the orange line and my dad is the green line.  This is from 2013 when my father started off very, very slowly.  Through 3 rounds we were tied, and then I took and held the lead at the end of the 4th round and held the lead through the middle of the 6th round when my dad went bonkers and made a bunch of birdies and picked up pars on holes he hadn’t gotten yet.  So what does it look like when someone is getting crushed in this competition?


After two full rounds, my father is 10 full strokes ahead of me.  It’s an absolute blow out through 2 rounds.  That black dot is a birdie, which pops made on the 18th hole of the second round today.  An early dagger.  Though I have plenty of holes that are currently sitting at double bogey, so I should make up ground fairly quickly (hopefully, anyway).  I’ll update this tomorrow after we’ve completed our rounds.



Update: Through three rounds: I’m down 75-81.


Update 2: Through 4 rounds I’m down 75-78 (and I do not have the tie breaker).


I play chicken with men on the street

Originally posted on mathbabe:

Lately you’ve seen a string of articles about how women say sorry too much. We’ve got yesterday’s New York Times opinion piece entitled Why Women Apologize and Should Stop, we’ve got Amy Schumer’s amazing-as-always skit on accomplished women apologizing for everything including existing, and even academics are weighing in.

This is not my problem. I don’t apologize as an automatic response. I learned that early on, when I got my first teaching evaluation; I had apologized exactly once to my (all female) class about not being prepared, in a summer semester where I met with them daily for 10 weeks, and at the end of the summer they all mentioned that I came to class unprepared. I had never come unprepared a second time, nor had I ever mentioned being unprepared a second time. That experience cured me of apologies more generally.

But I remain curious about how men…

View original 778 more words

Hannah and her sweets

Originally posted on God plays dice:

Apparently students in the UK have been protesting against the following question on a GCSE math exam (see e. g. coverage at The Guardian):

There are n sweets in a bag. Six of the sweets are orange. The rest of the sweets are yellow. Hannah takes a random sweet from the bag. She eats the sweet. Hannah then takes at random another sweet from the bag. She eats the sweet. The probability that Hannah eats two orange sweets is 1/3. Show that n²-n-90=0.

The probability that the first sweet is orange is $latex 6/n$. Now there are five orange sweets left out of $latex n-1$, so the probability that the second sweet is orange, assuming that the first one is, is $latex 5/(n-1)$. Therefore we need to solve $latex (6/n) times (5/(n-1)) = 1/3$. Multiplying it out gives

$latex {30 over n(n-1)} = {1 over 3}$

and we can…

View original 455 more words