Football Outsiders and field goal kicking: Update
First of all, I stand by everything I said in my previous post about Football Outsiders, but I guess there is more to add. Turns out that there is a student here who used to intern for Football Outsiders, and he pointed me to the methods section on their website where they say this:
Field goal kicking is measured differently. Measuring kickers by field goal percentage is a bit absurd, as it assumes that all field goals are of equal difficulty. In our metric, each field goal is compared to the average number of points scored on all field goal attempts from that distance over the past 15 years. The value of a field goal increases as distance from the goal line increases.
I agree completely with this. But you need to mention this fact or at least link to it even when you are describing the basics. Schatz tweeted this, among other things, to me (thanks for the mention):
That’s not it at all. There is a difference between simplifying your explanation of a complex statistical concept to a non-expert lay audience and simplifying your analysis to a point that renders your conclusions meaningless. Leaving out the fact that you are controlling for the distance of the field goal shouldn’t disappear when they are trying to write a simple summary. It’s an essential piece of the analysis, without which, renders all of your conclusions about place kicking meaningless. And in reading just the FO Basics it’s not clear at all that they are actually controlling for distance or anything else.
Maybe this is my fault for not reading more about the methods, but I tend to think that the onus is on Football Outsiders to make it clear, even in their simple summary, that they are controlling for field goal distance. But there is still the issue of why this wasn’t mentioned in the NY Times article either.
Schatz argues that they left the fact that they were controlling for field goal distance out of the NY times article for the purpose of simplicity. Or in his tweeted words:
Sure, I understand that things have to be simplified for a larger audience, but this is the NY Times. They published this article, for example, in 2007 about statistics that are misleading when you don’t properly control for explanatory variables. So, I think it would have been alright to explain in the NY Times that kicking distance is controlled for because without controlling for kicking distance, the conclusions are meaningless. We’re also talking about the same NY Times that hosted Nate Silver’s blog until recently!
Finally, even when controlling for field goal distance (and possibly other factors), I’m still not convinced that the ability of a place kicker varies randomly from year to year, though I don’t have any hard analysis (right now) to back this up. Though, Mike Lopez pointed me to this article from Sloan analyzing field goal kicking, and if you look at Table 5 there is some evidence that kickers’ abilities, at least some kickers’ abilities, are consistent from year you year. If kicker ability was changing dramatically from year you year we wouldn’t expect to see Janikowski twice in the top five best seasons or see Kris Brown twice in the bottom five seasons. This is certainly not proof that kicker’s performance from year to year is highly variable, but it also doesn’t support that argument either.
I would be very interested to see the full analysis that Football Outsiders performed to reach this conclusion, as I think it would be interesting to try and reproduce this.
P.S. I will once again state for the record, that I enjoy Football Outsiders and read it regularly.
Posted on October 8, 2013, in Uncategorized. Bookmark the permalink. 5 Comments.
So 2 lengthy blog posts devoted to the fact that FO doesn’t hit you over the head with the fact that they adjust for FG length? I’m not a fan of FO, but even I (an infrequent reader) certainly knew they adjusted for FG length — as you should have, if you are a regular reader. They try to adjust for everything, and FG length is one of the first and easiest things you would try to adjust. An adavnced kicking game rating system using raw FG accuracy would be one of the more absurd things imaginable. Actually, a distant second (in absurdity rank) to a blog post trying to “debunk” a straw man kicker evaluation process.
They were trying to make the argument that field goal kickers are inconsistent from year to year by citing Vanderjagt’s raw percentage of made field goals. That’s what it says in their NY times article, and that’s what it says in their FO Basics. I’m supposed to assume that they are adjusting for field goal distance when they themselves are supporting their argument with statistics that are clearly NOT adjusted for distance?
Let me remind you again how they phrase this theory: “Field-goal percentage is almost entirely random from season to season, while kickoff distance is one of the most consistent statistics in football.” “Field-goal percentage” is not a statistic that is adjusted for distance.
Thanks for reading.
I don’t know what they (football outsiders) do but I’ve been of the thought that there must be some reasonable way to adjust. It may require going a bit model-happy but so it goes.
IMO, my way of doing things would be to use the collective of all field goal locations as the evaluation bed (ignoring that kickers may be used relative to their ability… i.e. if you can’t hit a 40 yarder a team wouldn’t attempt them… which would skew things.)
Lacking anything other than easily available info (miss or not from published locations) I’d set up some kind of semi-parametric regression form w random effects. Such a model would capture a general trend pattern to fill in the gaps while letting the player speak for themselves otherwise.
Obviously, these are complicated answers to problems that people would otherwise want easy solutions, but so it goes. You’ll need something to construct provides a means of adjustment.
I’ve actually done this analysis, and I’m writing it up this weekend. I think I’ve got some interesting stuff.
Pingback: An exploration of Football Outsider’s theory that field-goal percentage is random from year to year: A full analysis | Stats in the Wild