I like the below quote form this article:

Michele Pellizzari, an economics professor at the University of Geneva in Switzerland, has a more serious claim: that course evaluations may in fact measure, and thus motivate, the opposite of good teaching.

His experiment took place with students at the Bocconi University Department of Economics in Milan, Italy. There, students are given a cognitive test on entry, which establishes their basic aptitude, and they are randomly assigned to professors.

The paper compared the student evaluations of a particular professor to another measure of teacher quality: how those students performed in a subsequent course. In other words, if I have Dr. Muccio in Microeconomics I, what’s my grade next year in Macroeconomics II?

Here’s what he found. The better the professors were, as measured by their students’ grades in later classes, the lower their ratings from students.

Cheers.

Until we acknowledge that the most common reason a method is chosen is because, “I saw it in a widely-cited paper in journal XX from my field” it is likely that little progress will be made on resolving the statistical problems in science.

From article by Jeff Leek.

On April 17th, the NY Times published an article entitled: “At a Long Island Beach, Human Tempers Flare Over Claws and Feathers”.  In this article, they state:

The fight comes amid growing concern nationwide about the impact of feral or stray cats on wildlife in general and birds in particular. Federal researchers have estimated that cats, including outdoor house cats and tens of millions of strays, kill 2.4 billion birds annually in the contiguous United States.

Where did that 2.4 billion number come from?  The article sites another article.  In that article they state:

In a report that scaled up local surveys and pilot studies to national dimensions, scientists from the Smithsonian Conservation Biology Institute and the Fish and Wildlife Service estimated that domestic cats in the United States — both the pet Fluffies that spend part of the day outdoors and the unnamed strays and ferals that never leave it — kill a median of 2.4 billion birds and 12.3 billion mammals a year, most of them native mammals like shrews, chipmunks and voles rather than introduced pests like the Norway rat.

The “report” that they cite can be found here.  It is an article in the journal Nature Communications called “The impact of free-ranging domestic cats on wildlife of the United States”.  You might remember this from articles such as this, this, this, this and, more recently, this.

I got involved in this when I was asked to review the paper for Alley Cat Allies.  (Full disclosure, I was paid by them to review the Nature Communications paper and my full review can be found here.) In general, I found the entire paper unsuitable for publication in an academic journal as a result of the numerous major statistical flaws.  I’m not alone in this belief.  You can find others who question the validity of the studies here and here, for instance.

But if you’re looking for more massive bird death “estimates”, don’t worry. It appears that Loss didn’t stop at just estimating cat predation mortality.  He has gone on to publish a whole series of papers (and landed a job at Oklahoma State) “estimating” bird mortality of different sorts:

• Refining estimates of bird collision and electrocution mortality at power lines in the United States: 12 – 64 million
• Estimation of Bird-Vehicle Collision Mortality on U.S. Roads: 89 – 340 million
• Estimates of bird collision mortality at wind facilities in the contiguous United States: 140,000 – 328,000
• Bird–building collisions in the United States: Estimates of annual mortality and species vulnerability: 100 million – 1 billion
• The impact of free-ranging domestic cats on wildlife of the United States: 1.3 – 4 billion

Those numbers are huge!  And I don’t trust a single one of them.

Cheers.

So I was trying to find an article with statistics in it that I could have my intro stats students reproduce as an assignment.  So I went to PloS Medicine and randomly chose a recent paper.  I can across this one: http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1001807#pmed-1001807-t001

I was going to have my student reproduce the numbers in this paragraph in the results section:

As described above, GII VLPs were distinct from the GI VLPs at 7 d post-vaccination. Interestingly, the GII.4C VLP (representing the vaccine component) associated much more closely with the early GII.4 VLPs than with either the late GII.4 VLPs or the GII VLPs from other genotypes (ANOVA: F_2,4 = 14.74, p = 0.0143), showing that the IgG response to GII.4C vaccination is highly cross-reactive with the early GII.4 VLPs but not the antigenically distinct contemporary GII.4 VLPs or other GII VLPs. Even as late as 180 d post-vaccination (Fig. 9G–I), clustering by genogroup (or by subclades within genogroups) remained (t-test: t₅₃ = 23.96, p < 0.001), although overall distances (dissimilarities) in IgG responses differentiating between VLPs had decreased compared to day 7. Specifically, the GI viruses clustered relative to the other virus strains (t₂₅ = 11.32, p = 0.0024), GII.4C continued to cluster with GII.4.1997 and GII.4.2002 (t₂₅ = 8.973, p = 0.0061), and the contemporary GII.4 VLPs remained tightly clustered with the other GII genotypes (t₃₈ = 19.24, p < 0.001). Interestingly, and echoing our findings above, GII.4.1997 remained somewhat distinct from GII.4C and GII.4.2002 (Fig. 9G–I) because of the elevated levels of IgG against GII.4.1997 VLP still found in several of the vaccine recipients 6 mo post-vaccination.

…but I can’t reproduce the numbers myself.  I can reproduce the p-value for the ANOVA (0.0143), but I can’t reproduce any of the p-values for the t-tests.  For instance, with a t-test statistic of 8.973 with 25 degrees of freedom, I get a two-sided p-value of 0.00000000272 as opposed to the stated p-value of 0.0061.  Am I missing something obvious?  Or are these p-values all wrong?

Follow-up question: If (IF!) the p-values are all wrong, as I suspect, should I still assign it to my students and ask them to reproduce the numbers as a lesson that they shouldn’t trust every statistics they see?

Cheers!

Final Four: Kentucky, Wisconsin, Duke, Michigan State

Kentucky beats Duke: Yosarian, ZachB, 7

Duke beats Kentucky: Pookie, Juho, 4

Kentucky beats Michigan St: Yosarian, ZachB, 13

Michigan State beats Kentucky: Juho, One Shining MGF, 2

Wisconsin beats Duke: Monte, ZachB, 6

Duke beats Wisconsin: ZachB, Monte, 4

Wisconsin beats Michigan State: ZachB, Wally, 15

Michigan State beats Wisconsin: Wally, ZachB, 3

We. Are. Still. Alive.

Disclaimer: I’m not guaranteeing that everything here is correct.  Please verify everything for yourself if you plan on using this information to hedge.

Scenarios where we finish first

There are only 32 possible scenarios left for the rest of the NCAA tournament.  There are two scenarios that would make us repeat champions.

1.) Final Four of Kentucky, Wisconsin, Gonzaga, and Michigan State with Gonzaga beating Kentucky in the finals.

2.) Final Four of Kentucky, Wisconsin, Gonzaga, and Louisville with Gonzaga beating Kentucky in the finals.

Scenarios where we finish second

There are three scenarios that we finish second in:

1.) Final Four of Kentucky, Wisconsin, Gonzaga, and Louisville with Gonzaga beating Wisconsin in the finals.

2.) Final Four of Kentucky,Wisconsin, Duke, and Louisville with Louisville beating Kentucky in the finals.

3.) Final Four of Kentucky,Wisconsin, Duke, and Michigan State with Michigan State beating Kentucky in the finals.

Some notes

• If Duke wins tomorrow, we cannot repeat as champions.
• If Gonzaga beats Kentucky in the finals, we win.
• If Kentucky makes it to the finals and loses, the worst we can finish is 4th.
• If Louisville wins tomorrow, ZachB cannot win.
• No matter what happens in the games tomorrow, we will still have a scenario to finish top 2.
• At the end of the Sunday night, we will either be in 6th, 12th, or 14th.
• We are currently in 13th.
• 7 of the top 12 teams have no chance to cash.
• We cannot win if Kentucky wins it all.
• If Gonzaga beats Duke, we end the night in 14th no matter what happens in the other game.
• If Louisville and Duke win, we end the night in 6th, but have no chance to finish first.
• The worst we can finish at the end of the tournament is 18th.  This would happen if Kentucky, Wisconsin, Duke and Louisville made it to the final four and Wisconsin beat Louisville.
• In 7 of the 32 scenarios remaining, we finish in third, with a remaining 4 scenarios where we finish in fourth.
• In 24 out of the 32 scenarios we finish in the top 10.
• Based on current odds, the Final Four is expected to be Kentucky, Wisconsin, Duke, and Michigan State with Kentucky beating Duke in the finals.  In that scenario we finish 7th.

There are 4 possible final four scenarios.  Each has its own heading.  Under each of the 4 scenarios, I list a finals result, and, given that result, who finishes first, second, and what place my team finishes.

Potential Winners with 6 games to play

Final Four: Kentucky, Wisconsin, Duke, Louisville

Kentucky beats Duke: Yosarian, ZachB, 8

Duke beats Kentucky: Juho, Pookie, 3

Kentucky beats Louisville: Yosarian, Bayz, 16

Louisville beats Kentucky: Juho, One Shining MGF, 2

Wisconsin beats Duke: Monte, Wally, 7

Duke beats Wisconsin: Monte, ZachB, 4

Wisconsin beats Louisville: Monte, Wally, 18

Louisville beats Wisconsin: Wally, Monte, 3

Final Four: Kentucky, Wisconsin, Duke, Michigan State

Kentucky beats Duke: Yosarian, ZachB, 7

Duke beats Kentucky: Pookie, Juho, 4

Kentucky beats Michigan St: Yosarian, ZachB, 13

Michigan State beats Kentucky: Juho, One Shining MGF, 2

Wisconsin beats Duke: Monte, ZachB, 6

Duke beats Wisconsin: ZachB, Monte, 4

Wisconsin beats Michigan State: ZachB, Wally, 15

Michigan State beats Wisconsin: Wally, ZachB, 3

Final Four: Kentucky, Wisconsin, Gonzaga, Louisville

Kentucky beats Gonzaga: Yosarian, Lavarel, 5

Gonzaga beats Kentucky: One Shining MGF, Coop, 1

Kentucky beats Louisville: Lavarel, Yosarian, 15

Louisville beats Kentucky: Juho, Wally, 3

Wisconsin beats Gonzaga: Wally, ur1, 6

Gonzaga beats Wisconsin: Wally, One Shining MGF, 2

Wisconsin beats Louisville: Wally, Monte, 16

Louisville beats Wisconsin: Wally, Monte, 4

Final Four: Kentucky, Wisconsin, Gonzaga, Michigan State

Kentucky beats Gonzaga: Lavarel, ZachB, 7

Gonzaga beats Kentucky: One Shining MGF, ZachB, 1

Kentucky beats Michigan State: Lavarel, ZachB, 13

Michigan State beats Kentucky: Yosarian, Juho, 3

Wisconsin beats Gonzaga: Wally, ZachB, 7

Gonzaga beats Wisconsin: ZachB, Wally, 3

Wisconsin beats Michigan State: Wally, Zach B, 15

Michigan State beats Wisconsin: Wally, ZachB, 3

Arizona -1.5

Kentucky -11

Gonzaga +2.5

Louisville +2.5

Cheers.

If I had to bet every game (Games I really like in BOLD):

Oklahoma +2.5

Utah +6

Louisville -2.5

Gonzaga -8.5

Xavier +10.5

West Virginia +13

North Carolina +6

Notre Dame +1.5

Cheers.

If I had to bet every game.  Bets I really like in bold:

San Diego State +9

Gonzaga -6

Oregon +12

Maryland EVEN

Wichita State +1.5

Oklahoma -4.5

Louisville +2.5

Cheers.