• I was reading Flowingdata.com this morning and the blog post at the top of the page was about privacy.  Nathan Yau says:

    With all the stuff going on with surveillance and data privacy — especially the past week — it’s worthwhile to revisit this essay by Daniel J. Solove, a professor of law at George Washington University, on why privacy matters even if you “have nothing to hide.”

    So, I went and read that article where they talk about the argument “nothing to hide, nothing to worry about.”  It seems obvious to me that this argument is weak, but in case you need some quick counter arguments, here is a list of response from that article:

    • My response is “So do you have curtains?” or “Can I see your credit-card bills for the last year?”
    • So my response to the “If you have nothing to hide … ” argument is simply, “I don’t need to justify my position. You need to justify yours. Come back with a warrant.”
    • I don’t have anything to hide. But I don’t have anything I feel like showing you, either.
    • If you have nothing to hide, then you don’t have a life.
    • Show me yours and I’ll show you mine.
    • It’s not about having anything to hide, it’s about things not being anyone else’s business.
    • Bottom line, Joe Stalin would [have] loved it. Why should anyone have to say more?

    These are all great counter arguments to the “nothing to hide, nothing to worry about” argument, but I’d like to add another: Privacy is a basic human right. I realize that sounds dramatic, but go read the Universal Declaration of Human Rights for yourself.

    Article 12: No one shall be subjected to arbitrary interference with his privacy, family, home or correspondence, nor to attacks upon his honour and reputation. Everyone has the right to the protection of the law against such interference or attacks.

    And why are human rights important?  From the preamble:

    Whereas recognition of the inherent dignity and of the equal and inalienable rights of all members of the human family is the foundation of freedom, justice and peace in the world.

    So, according to the United Nations, one of your basic human rights is your right to privacy, and human rights are the foundation of freedom, justice, and peace. That’s a big deal. A really big deal. Let me repeat this one more time: Your privacy is a big deal.

    The UN does add the caveat of “arbitrary interference” because of course there are situations where you lose the right to privacy. So what does arbitrary mean? Just to be sure I looked up the definition. Google returns two definitions. The first is “Based on random choice or personal whim, rather than any reason or system” and the second is “(of power or a ruling body) Unrestrained and autocratic in the use of authority.” From what little I have read about the NSA privacy stuff, it sounds like these definitions are describing exactly this situation. The “51% sure” that a person is foreign sure sounds a lot like “personal whim” to me. Whether or not this data collection is an “arbitrary interference” is a debate that we need to have as a society. Where is the line?

    The issue of privacy and where this line is is particularly interesting to me because when I was in graduate school I wrote my dissertation on statistical disclosure control. This involves attempting to balance the dissemination to researchers of useful data against the privacy of the individual. In terms of what I was studying, the benefit of data dissemination is useful research, often on topics of public health, whereas in the NSA situation the alleged benefit is in fighting terrorism. In both cases, the risk is the erosion of individual privacy. In public health neither extreme is an acceptable solution. Either you release none of the data and you have perfect privacy at the cost of halting scientific research, or you release all of the data to anyone who wants to use it for research (or any other purpose!) and you achieve maximum utility at the cost of total lack of privacy. Some balance between these two must be struck. I think it’s probably the same with the NSA and the government: Some balance must be struck between fighting terrorists, who are a real and legitimate threat, and the ideals that we have as citizens of the United States and as human beings.

    One of the difficulties in having this debate, and again I’ll make a comparison to my research, is that quantifying utility and privacy in many situations is very difficult. This makes it hard to really quantify the costs and benefits associated with decisions on where to draw the line when releasing data to researchers. This is likely even more difficult to quantify with what the NSA is doing. If we could easily measure how much terrorism is prevented versus how much privacy we are losing, society could have a debate and reach some sort of conclusion. But we don’t have these measures. (One potential way to measure terrorism would be in attacks prevented, but it’s virtually impossible to do this with any accuracy. And measuring privacy might be even more challenging.) If this were an economic problem we could weigh the costs versus the benefits on some monetary scale. If this were drug testing we would weigh the benefits of the drug versus the side-effects. But in this situation there is no clear way to measure either of these concepts in an objective way, and even if we could, everyone has a different idea about where the bar should be set.

    So ultimately, as a society, we’re in a situation where we need to have a debate about privacy protection versus terrorism prevention where both sides have important concerns that are often in conflict with one another. We need to decide where to tip the scale so that we balance these two things in a meaningful way that respects our American and international ideals of freedom, justice, and peace. But we need to do this without using any objective scale. It’s a an extremely difficult problem and it’s just going to get more complicated as we move further into a world dominated by data.

    Cheers.

  • This is incredible: “Crosby had never been held without a point in any four consecutive NHL games, regular or postseason, until the Bruins series.”

    statsbylopez's avatarStatsbyLopez

    Image

    Some quick notes to wrap up the Boston Bruins’ recent sweep of the Pittsburgh Penguins. Boston held Pittsburgh stars Sidney Crosby and Evgeni Malkin off the board, while as a team, the Penguins managed just two goals in four games (all data taken from hockey-reference.com).

    1- Penguins star Sidney Crosby was held without a point in four games. Here are Crosby’s longest scoreless game streaks in his NHL career

    05-06: 3 games
    06-07: 3 games
    07-08: 1 game
    08-09: 3 games
    09-10: 2 games
    10-11: 2 games
    11-12: 2 games
    12-13: 1 game

    In other words, Crosby had never been held without a point in any four consecutive NHL games, regular or postseason, until the Bruins series.

    1a) In the 2008-09 season, Crosby went three games without a point in the final three games of the Stanley Cup finals against Detroit. That streak should come with an asterisk, as Crosby was…

    View original post 453 more words

  • Updated June 7, 2013 at 12:34am

    Total Prevention is a measure of runs prevented weighted by the number of batters they have faced (Higher is better).

    Yearly Expected ER  is approximately the expected number of runs that a team would allow is that pitcher pitched every inning of every game for a team (Lower is better).

     
    Rank Pitcher Team Total Prevention Yearly Expected ER
    1 ClaytonKershaw LAD 697 318
    2 MattHarvey NYM 678 310
    3 AdamWainwright STL 672 343
    4 ClayBuchholz BOS 585 344
    5 CliffLee PHI 585 377
    6 MaxScherzer DET 561 342
    7 YuDarvish TEX 502 394
    8 FelixHernandez SEA 486 415
    9 ShelbyMiller STL 470 372
    10 JordanZimmermann WSN 469 406
    11 HisashiIwakuma SEA 451 413
    12 ChrisSale CHW 450 382
    13 AnibalSanchez DET 449 400
    14 A.J.Burnett PIT 443 417
    15 MikeMinor ATL 426 406
    16 PatrickCorbin ARI 425 419
    17 JamesShields KCR 387 468
    18 LanceLynn STL 385 425
    19 JustinMasterson CLE 369 472
    20 TravisWood CHC 369 441
    21 JeffSamardzija CHC 356 470
    22 StephenStrasburg WSN 356 457
    23 MadisonBumgarner SFG 351 468
    24 DerekHolland TEX 318 480
    25 JonLester BOS 292 504

    Full Rankings

  • Updated June 7, 2013 at 12:34 am.

    Yearly Production for a player is the approximate number of runs a team would score if the entire lineup consisted of the same player.

    Total Production is weighted by the number of plate appearances.

     
    Rank Player Total Production Yearly Production
    1 MiguelCabrera 759 1591
    2 ChrisDavis 747 1752
    3 JoeyVotto 693 1420
    4 PaulGoldschmidt 581 1338
    5 CarlosGonzalez 532 1176
    6 Shin-SooChoo 488 1061
    7 TroyTulowitzki 467 1235
    8 MikeTrout 463 959
    9 JoshDonaldson 454 1029
    10 JoeMauer 450 1024
    11 DustinPedroia 442 920
    12 JustinUpton 429 993
    13 PrinceFielder 429 944
    14 DexterFowler 422 968
    15 JoseBautista 419 1004
    16 JeanSegura 418 993
    17 MichaelCuddyer 414 1295
    18 EdwinEncarnacion 405 873
    19 EvanLongoria 402 896
    20 RyanBraun 400 982
    21 RobinsonCano 398 887
    22 MarkTrumbo 395 845
    23 MannyMachado 393 811
    24 DavidOrtiz 391 1289
    25 BryceHarper 391 1194

    Full Rankings

  • StatsInTheWild MLB rankings as of June 7, 2012 at 2pm.  SOS=strength of schedule

    Team Rank Change Record ESPN TeamRankings.com SOS Run Diff
    St. Louis 1 ↑2 39-12 1 1 24 +40
    Cincinnati 2 ↑3 36-24 3 7 20 +29
    Boston 3 ↓2 37-24 4 2 9 +23
    Texas 4 ↑3 36-23 2 3 22 +21
    Pittsburgh 5 ↑3 35-25 6 12 17 +25
    Detroit 6 ↑4 32-26 8 9 21 +24
    Tampa Bay 7 ↑8 32-27 12 8 7 +28
    Oakland 8 ↑6 37-25 7 5 15 +4
    Baltimore 9 ↓5 34-26 9 11 6 +17
    Atlanta 10 ↓4 37-23 5 4 30 +22
    Arizona 11 ↓2 34-26 11 10 23 +3
    NY Yankees 12 – 35-25 10 6 5 +7
    Colorado 13 ↓11 32-29 15 14 25 +8
    Cleveland 14 ↑5 30-29 13 13 12 +17
    SF 15 ↓4 31-28 14 15 18 +6
    Kansas City 16 ↓3 25-32 24 20 8 +1
    Minnesota 17 – 26-31 21 18 16 -2
    Toronto 18 ↑8 25-34 26 16 1 -10
    Chi Cubs 19 ↑4 24-33 23 23 10 -1
    San Diego 20 ↑4 28-32 17 21 19 -12
    LA Angels 21 ↑7
    26-34 18 25 11 +2
    Washington 22 ↓2 29-30 16 17 28 -24
    Chi WSox 23 ↑2 25-33 20 26 13 -13
    Seattle 24 ↑3 26-35 25 22 4 -28
    Philadelphia 25 ↓7 31-30 19 19 29 -13
    Milwaukee 26 ↓10 22-37 28 27 3 -35
    LA Dodgers 27 ↓5 26-33 22 24 14 -28
    Houston 28 ↑1 22-39 29 28 2 -24
    NY Mets 29 ↓8 23-33 27 29 27 -42
    Miami 30 – 16-44 30 30 26 -45

    Past Rankings:

    4/24/2013

    4/17/2013

    4/12/2013

    9/25/2012

  • Perhaps Mac Engel is a brilliant satirist or perhaps he is just trying to get people to pay attention to him.

    As a statistician, I’m going to make an assumption (that’s what us stats guys do), and I’ll start by assuming that Engel is indeed sincere in his belief that “Mathematics is not a friend of baseball” in his article of the same title. Assuming this is true, it seems clear to me that he doesn’t really have any idea what he is talking about.

    Engel’s article opens with some jokey, over-the-top claims about math in sports (number-crunching will ruin your kid’s dodgeball game!). But his overall point appears to be in earnest. In his second paragraph (if you can call anything in this a paragraph; it’s more collection of individual sentences) he says (emphasis added):

    Scores of math whizzes, nerds and live-in-their-parent’s-basement geeks are threatening to turn Royals at Rangers into a Bobby Fisher vs. Boris Spassky chess match, minus the intellect.

    This sentence demonstrates clearly that Engel does not understand the nature of statistics today and how they are applied to baseball.

    First, the phrase “live-in-their-parent’s-basement geeks” is being used derogatorily here to indicate that statisticians are losers unable to function in society, and are thus relegated to their parents’ basements because of their personal and professional failures. This was my first indication that Engel has no idea what he’s talking about. These “live-in-their-parent’s-basement” geeks surely exist , but in today’s world these geeks more often come in a different variety: the employed-in-a-recession-six-figure-income kind. (There’s also the rock-star-stats-geek like Nate Silver, who is more important to the New York Times’ website traffic than most of its regular journalists.) While journalism jobs are rapidly disappearing, jobs based on data are expanding so rapidly that there are not enough people with the skills to fill these positions.  Nearly all of the successful businesses in the world know the value of data (see: Book, Face) and you can argue that one of the big differences between Obama and Romney in the last election was how the two campaigns handled, managed, and took action based on data and analysis (Obama embraced it, Romney largely ignored it; how did that work out?). The point is, the best way to make decisions is when those decisions are based on data and appropriate analysis. Baseball is a game where many decisions must be made, and those decisions can best be made when they are aided by data.

    Secondly, Engel seems to fail to grasp a major difference between chess and baseball: chess is deterministic; baseball is not. Every time a chess player moves Bg5, the bishop gets there with certainty, whereas in baseball, not every routine fly ball will get caught (and sometimes they bounce off your head and go over the fence). He correctly notes that this uncertainty is what makes baseball exciting. As he says: “Math is never wrong. Baseball very much is, which is why I love it.” I’m not sure exactly what he means by “math is never wrong,” but when he says “baseball very much is,” I assume he is implicitly referring to random event outcomes. And this is precisely why math, specifically statistics and probability, is useful in making decisions in baseball. Statistics and probability give us a principled way to quantify uncertainty about events that have occurred in the past and events that will happen in the future. Statistics can’t say what is going to happen on any given play with certainty (e.g., Miguel Cabrera will hit a home run in this at-bat), but it can help someone to make statements about how likely an event is to occur (e.g., Miguel Cabrera is more likely to hit a home run in an at-bat than Norichika Aoki). And just because we can’t say with certainty what will happen on any given trial doesn’t mean that it isn’t useful. There are entire companies that exist based on events that might only occur 1 in 1000 times (think click-throughs on an ad, or responses to fake Viagra emails).

    So what numbers and maths are specifically ruining baseball in the opinion of Engel? Is it the often cited and complicated wins above replacement (WAR) statistic? Nope. Is it the opaque Ultimate Zone Rating (UZR)? No, not that either. The entire rest of the article is about how math has ruined the game and made us a “prisoner to all of these bleepin’ numbers,” and his example is: PITCH COUNTS!


    Pitch counts are an example of math in baseball in the same way that page numbers are an example of math in literature. If you think numbers and math are ruining baseball, which I disagree with thoroughly, I’m sure you can make a reasonable argument, but you need to be more persuasive than talking about pitch counts. There is a reason the Oakland A’s won all those regular season games with their tiny payroll, and it wasn’t simply luck. They were making better decisions that everyone else at the time because they were using data in a better way than their opponents, and it gave them an advantage in the long run.

    So what I’m trying to say is that math and statistics aren’t ruining baseball. Now if you’ll excuse me I’ve got to go do the dishes. I can hear my mother yelling at me from upstairs.

  •  

    I’m still blocked from following @BillBarnwell on Twitter.  But what I don’t quite understand is what is the point of blocking anyone if your account is public?  I’m blocked from following him, but I can still read all his tweets because his account is public.  So, what do you gain by blocking someone?  There must be some benefit to it?  Is it just that I can’t direct message him?  Screen shot 2013-06-02 at 7.02.11 PM

     

    Cheers.

  •  

    Deadspin answers the question of whether you live in IHOP America or Waffle House America.

    Screen shot 2013-06-02 at 9.22.16 AM

    Screen shot 2013-06-02 at 9.26.53 AM Screen shot 2013-06-02 at 9.26.41 AM

     

    I also found it particularly interested that at the very end of the Deadspin article they say:

    Waffle House does so much business in the South that the impact of hurricanes on Waffle House service is actually an informal measure of hurricane intensity.

    Cheers.

  • David McRaney's avatarYou Are Not So Smart

    The Misconception: You should focus on the successful if you wish to become successful.

    The Truth: When failure becomes invisible, the difference between failure and success may also become invisible.

    In New York City, in an apartment along the Hudson River, above trees reaching out over sidewalks and dogs pulling at leashes and conversations cut short to avoid parking tickets, a group of professional thinkers once gathered and completed equations that would both snuff and spare several hundred thousand human lives.

    People walking by the apartment at the time had no idea that four stories above them some of the most important work in applied mathematics was tilting the scales of a global conflict as secret agents of the United States armed forces, arithmetical soldiers, engaged in statistical combat. Nor could people today know as they open umbrellas and twist heels on cigarettes, that nearby, in an apartment overlooking Morningside…

    View original post 6,331 more words