• I attended a session at JSM (in Miami.  In August….) where the big topic was statistical disclosure control (SDC) via suppression in linked tables.  As far as I can tell suppression is the most popular and widely used method of SDC for tables, but it seems that the application of this procedure is extremely ad hoc.  Data disseminating organizations all have different rules for when they feel that a cell count or total is too small to be released, but it seems that this is all done by educated guessing as to what cell values are unsafe.  I wonder if there are any formal guarantees that can be provided to individuals or organizations whose data is being disseminated in tables and protected via suppression?  Certainly tables with sensitive values cannot simply be released to the public and something needs to be done to address the problem.  Cell suppression is something and it certainly offers something in the way of protection (what is that something?).  But I don’t know that cell suppression is more about an appearance of privacy (which is still very important), rather than actually providing privacy.

    Of course, I’d love someone to respond to me and tell me that I am dead wrong and put my mind at ease.

    Cheers.

     

     

     

  • I don’t know how many of you have ever been to Miami Beach in August, but it’s not exactly…..comfortable.  But that’s where my quest for knowledge took me in the first week of August to the Joint Statistical Meetings (JSM) 2011.

    I attended a really interesting session “Statistical Analyses of Judging in Athletic Competitions: The Role of Human Nature“.  I missed the first talk about racial bias in Major League Baseball (MLB) umpires, but I caught the last three, which were all very interesting.  John Emerson, who organized the session, presented “Statistical Sleuthing by Leveraging Human Nature: A Study of Olympic Figure Skating”.  Ryan Rodenburg (Blog: Sports Law Analytics) presented his paper “Perception ≠ Reality: Analyzing Specific Allegations of NBA Referee Bias”. His approach was rather interesting.  Rather than try to look for overall biases in NBA referees, he attempted to validate or invalidate specific allegations levelled against specific referees.  His talk was followed by Kurt Rothoff who presented his paper “Bias in Sequential Order Judging: Primacy, Recency, Sequential Bias, and Difficulty Bias”, which focused on judging in gymnastics.  One of the key findings of this work was, from his abstract, “Contestants who attempt higher difficulty increase their execution score, even when difficulty and execution scores are judged separately.”  This makes me a little bit nervous because the better athletes are attempting the more difficult routines and, since they are better athletes, may receive higher execution scores because they are better athletes to begin with.  Is there anyone out there who has ever been a competitive gymnast who has any thoughts on this?

    Cheers!

  • I’ve been playing around with the faces function from the R package aplpack.  I haven’t used it in a while, but there are some new features that I’ve either never noticed before or they are new.  Color has been added to the faces and you can now plot the faces.  There is also the superfluously fantastic option of displaying the faces as Santa Claus.

    Here are some of my examples:

    Golf: Statistics from several of my friends collected via oobgolf.com.  (I’m SITW on the lower right.) The face is handicap, the mouth is scoring average, the eyes are average putts, the hair is the percentage of fairways hit, nose is greens in regulation (GIR), and ears are the total number of rounds you play. The faces are plotted with fairway percentage on the x-axis and GIR on the y-axis.

    Santa_Golf: Same golf data with Santa option.

    NFL2010: Final NFL regular season team statistics.  The face represent the offense and the defense is represented by hair. The size of the nose indicates sacks, the ears indicate turnovers (ear width is interceptions; ear height is forced fumbles).  The eyes indicate penalties and, finally, the size of the mouth indicates wins with a smiling face if the team made the playoffs (a really nice touch, if you ask me.)  The face at the bottom right indicates the league leader.

    Some observations on the NFL faces:  The two superbowl teams last year (Pittsburgh and Green Bay) are both located at the bottom of the graph and there faces look very, very similar.  San Diego looks similar to to both Green Bay and Pittsburgh (similar face, nose, eyes, and hair), but the big differences are the ears and, of course, the San Diego face is frowning.  Another thing that pops out at me is how similar Houston and New England look to each other.  They have very similar face shape, eyes, and hair.  The big differences are the nose and ears (sacks and turnovers).

     

    Cheers.

    ##NFL CODE

    library(aplpack)

     

    x<-read.csv(“\StatsInTheWild\NFL2010.csv”,header=TRUE)

    x[33,]<-x[32,]

    x$abbr<-sort(c(“NE”,”NYJ”,”Mia”,”Buf”,”Pit”,”Bal”,”Cle”,”Cin”,”Ind”,”Jac”,”Hou”,

    “Ten”,”KC”,”SD”,”Oak”,”Den”,”Phi”,”NYG”,”Dal”,”Was”,”Chi”,”GB”,”Det”,”Min”,”Atl”

    ,”NO”,”TB”,”Car”,”Sea”,”StL”,”SF”,”Ari”,”ZZ”))

    x$abbr[27:28]<-c(“SF”,”Sea”)

    x$abbr[33]<-“League Leader”

    x$lab<-paste(x$abbr,x$W,sep=”: “)

    x$TOP<-as.numeric(substring(x$TOP.x,1,2))

    ##Playoff Teams: creating a playoff indicator

    rows<-c(2,3,6,12,14,16,19,20,22,24,25,28)

    x$playoffs<-rep(0,33)

    x$playoffs[rows]<-1

     

    ##Finding the league leader in all variables

    num<-sapply(x,is.numeric)

    x[33,num]<-sapply(x[,num],max)

    def<-c(6,22:23,26:29)

    x[33,def]<-sapply(x[,def],min)

    x$lab<-paste(x$abbr,x$W,sep=”: “)

    ##Defining the names

    names(x)[c(2,3)]<-c(“Wins”,”Losses”)

    names(x)[c(13,14,15,16)]<-c(“Off PPG”,”Off YPG”,”Off Pass”,”Off Rush”)

    names(x)[c(22,23)]<-c(“Penalties”,”Pen Yards”)

    names(x)[c(26:29)]<-c(“Def PPG”,”Def YPG”,”Def Pass”,”Def Rush”)

    names(x)[c(5:6)]<-c(“Points For”,”Points Against”)

    pdf(“/StatsInTheWild/NFL2010.pdf”,width=15,height=10)

    ##Columns used for plotting

    x<- x[order(x[,4]),]

    plot.cols<-c(5,6)

    ##Offense = face, Defense = hair, penalty= eyes, Wins and playoffs = mouth, turnovers = ears

    ##Columns used for faces: which columns am i going to use for the data

    col<-c(15,16,14,2,2,41,22,23,28,29,27,36,36,30,32)

    ##creating the faces without plotting them.

    a<-faces(x[,col],labels=x$lab,face.type=1,plot=FALSE)

    ##creating text for the legend

    g<-paste(a[[2]][,1],a[[2]][,2],sep=”: “)

    ##building the plot

    plot(x[,plot.cols],bty=”n”,xlim=c(200,600),main=”2010 NFL Season”)

    text(rep(540,15),seq(475,325,length.out=15),g)

    ##plotting the faces

    plot.faces(a,x[,plot.cols[1]],x[,plot.cols[2]],width=30,height=30)

    dev.off()

  • Tomorrow I defend my dissertation. If all goes well, you will all have the opportunity to finally call me doctor; I know you are just as excited about this as I am. Hopefully, I will have more time to write blog posts post-defense.

    Cheers.

  • Well, I’m almost done with my dissertation, which means I’m almost done with my Ph. D.  And when I say done, I mean it in both the senses of “finished” and “sick of”.  I have a nearly complete document AND a defense date.  Now all I have to do is put the most important skill I learned in grad school to good use: finding and filling out paperwork.  Anyone can write a 100+ page dissertation filled with original thoughts, but only the best and brightest can jump through all of the bureaucratic hoops to actually complete the degree.

    Anyway, I really enjoyed my dissertation topic, which, I hear, is not something that everyone experiences.  I’ll eventually come back to the topic (statistical disclosure limitation), but I really just need some time away from it.   I’ll get my wish as I’ll be starting a post-doc this summer researching statistical genetics, which I am probably a little over excited to start.

    Cheers.

  • I’m currently attending the 2011 ENAR spring meeting in Miami.  I arrived Sunday night an presented a poster at the opening poster presentation session.  On Monday, I attended two sessions in the afternoon: the survival analysis section and, later, the policy section.

    In the policy section, I saw a presentation entitled “Issues in the use of survival analysis to estimate damages in equal employment cases” by Qing Pan and Jooseph L. Gaswirth, which has been published in the journal Law, Probability, and Risk in the March 2009 issue.  The presentation was two-fold: First they presented some basic methods for determining whether or not discrimination had taken place.  In this case (age discrimination),  it was fairly evident that the infraction has occurred.  Second, the authors presented how to assess the compensation that should be awarded to the parties which had been discriminated against.  In order to do so, they applied survival analysis techniques to estimate how long someone would have worked at the company if they had been employed.  Very interesting stuff.


    Along the same legal lines, I happened to pick up a book called “A Very Short Introduction to Statistics” by David J. Hand.  While I was flipping through it I came across a section about a woman named Sally Clark.  She was a woman who had two children, both of whom died within the first 11 weeks of their respective lives.  Subsequently, she was charged with murder as it seemed suspicious that TWO of her children had both died so young.  During the trial, Professor Sir Roy Meadow, (famous for proposing the theory of Munchausen Syndrome by Proxy (MSbP)) claimed that the chances of two of her babies dying in this fashion totally by chance was 73,000,000:1.  At those odds, I suppose you would have to convict the person.  However, his method for arriving at this number was flawed.  The Royal Statistics Society issued a statement that began “In the recent highly-publicised case of R v. Sally Clark, a medical expert witness drew on published studies to obtain a figure for the frequency of sudden infant death syndrome (SIDS, or “cot death”) in families having some of the characteristics of the defendant’s family. He went on to square this figure to obtain a value of 1 in 73 million for the frequency of two cases of SIDS in such a family.”  (Read the whole statement here.)  The way I feel about this can be summed up by some comments my friend (a lawyer) made when I emailed him about this case: “That’s wild that that happened in 1999. I figured it would be like
    1899.”

    So anyway, I am now sitting in my hotel room at the Leamington (students can’t afford the Hyatt).  I’ll leave you with a picture of the hotel I am staying at.  I can’t wait to get a job.

    Cheers.



  • Watson, an IBM computer, recently competed on Jeopardy, demoralizing his opponents Ken Jennings and Brad Rutter. (Jennings describes what it was like to lose the Watson here.) Really it was an uneven playing field from the start. For example, “Watson’s brain showcases several IBM technologies. The hardware is jammed into 10 refrigerator-sized racks filled with Power7 server blades. To be exact, there are 90 Power750 servers filled with four processors each — and each processor has 8 cores, for a total of 2,880 cores altogether.” (InformaWorld) That’s a lot of cores for our tiny human brains to compete against. Therefore, I am calling for an all computer Jeopardy tournament. Similar to the netflix prize, different organization could compete in a tournament pitting their algorithms against other organizations’ algorithms with the winner collecting a large cash prize and valuable free publicity and marketing. Just imagine IBM’s Watson competing against Google’s BrinPage and Microsoft’s Gates. That would be pretty intense.

    Cheers.

  • 14 week of the NFL Season are gone. 3 weeks remain.

    There have been a few big changes in the projected playoff seeds. In the AFC, the Jets fall to a projected 6 seed and the Ravens take over the 5 seed. Also, Jacksonville and Kansas City change places with Jacksonville projected to be the three seed with Kansas City the 4 seed.
    In the NFC, Green Bay is out and the Giants are in as the six seed.

    Here are the updated SITW projected playoff seeds:
    View the full rankings here.

    Week 15 Playoff projections (through week 14):
    Playoff Projections:
    AFC Projected seeds (Expected Wins) [Probability they make playoffs]{Ranking}:
    1. New England (13.6973)[1]{1}
    2. Pittsburgh (12.5784) [.9999]{3}
    3. Jacksonville (9.9711) [.8196]{13}
    4. Kansas City (9.7525) [.6738]{16}
    5. Baltimore (11.3688) [.9992]{4}
    6. New York Jets (10.837) [.9088]{5}

    NFC Projected seeds:
    1. Atlanta (13.6973) [1]{2}
    2. Philadelphia (11.2133) [.9151]{8}
    3. Chicago (10.6212)[.9047]{10}
    4. St. Louis (7.3868) [.5744]{22}
    5. New Orleans (11.2239) [.9038]{6}
    6. New York Giants (10.6128) [.6983]{12}

    Cheers.

  • The interesting thing about these playoff projections are that Philadelphia is projected to be the 2 seed in the NFC over Chicago even though Chicago has 1 more win than them. Take a look at the rest of the Eagle’s schedule and then look at the Bear’s remaining schedule and it becomes very clear why these are the projections.

    You can view the full rankings here.

    Week 14 Playoff projections:
    Playoff Projections:
    AFC Projected seeds (Expected Wins) [Probability they make playoffs]{Ranking}:
    1. New England (13.6874)[1]{2}
    2. Pittsburgh (12.4096) [1]{3}
    3. Kansas City (10.425) [.843]{15}
    4. Jacksonville (9.518) [.7124]{14}
    5. New York Jets (11.8408) [.9998]{4}
    6. Baltimore (11.2332) [.9982]{5}

    NFC Projected seeds:
    1. Atlanta (13.6874) [1]{1}
    2. Philadelphia (10.9962) [.8438]{9}
    3. Chicago (10.587)[.8484]{10}
    4. St. Louis (7.4758) [.477]{23}
    5. New Orleans (11.091) [.8424]{6}
    6. Green Bay (10.3086) [.7648]{7}

    My Rankings: (Wins)
    Teams Wins
    1 Atlanta (10)
    2 NewEngland (10)
    3 Pittsburgh (9)
    4 NYJets (9)
    5 Baltimore (8)
    6 NewOrleans (9)
    7 GreenBay (8)
    8 TampaBay (7)
    9 Philadelphia (8)
    10 Chicago (9)
    11 Miami (6)
    12 Cleveland (5)
    13 NYGiants (8)
    14 Jacksonville (7)
    15 KansasCity (8)
    16 Tennessee (5)
    17 Indianapolis (6)
    18 Minnesota (5)
    19 SanDiego (6)
    20 Oakland (6)
    21 Washington (5)
    22 Seattle (6)
    23 StLouis (6)
    24 Houston (5)
    25 Dallas (4)
    26 SanFrancisco (4)
    27 Denver (3)
    28 Arizona (3)
    29 Cincinnati (2)
    30 Buffalo (2)
    31 Detroit (2)
    32 Carolina (1)

    Cheers.

  • First off, what a joke the NFC West is.
    Although, I am excited about rooting for a 6 win team to make the playoffs and get a first round home playoff game. Nice work NFL.

    A thought about being the 2 seed: I think in a year like this there may be some advantage to being the 2 seed rather than the 1 seed.

    Let’s look at the AFC. Let’s assume that my playoff projections hold true. The wild card winners seeds will either be (3,4), (3,5), (4,6), or (5,6). After the first round of playoffs, the one seed plays the lowest remaining seed. This has to be the 4, 5, or 6 seed. So they are going to have to play Indianapolis, New York Jets or Pittsburgh. I would argue that all three of these teams are better than the Kansas City Chiefs. The two seed has to play either the 3, 4, or 5 seed. They are guaranteed to not have to play the Steelers, and they avoid the Jets in all scenarios except where both the Jets and Steelers win. This looks to me like an easier path to the AFC championship game.

    Week 12 Playoff projections:
    Playoff Projections:
    AFC Projected seeds (Expected Wins) [Probability they win the Super Bowl]{Ranking}:
    1. New England (12.92)[.199]{2}
    2. Baltimore (11.90) [.12]{5}
    3. Kansas City (10.05) [.0008]{15}
    4. Indianapolis (8.88) [.0004]{14}
    5. New York Jets (12.45) [.1292]{3}
    6. Pittsburgh (11.77) [.1162]{4}

    NFC Projected seeds:
    1. Atlanta (13.45) [.349]{1}
    2. Philadelphia (10.86) [.0208]{10}
    3. Chicago (10.49) [.0114]{11}
    4. Seattle (7.29) [0]{22}
    5. New Orleans (10.86) [.0254]{6}
    6. Green Bay (10.15) [.0138]{8}

    My Rankings: (Wins)
    1 Atlanta (9)
    2 NewEngland (9)
    3 NYJets (9)
    4 Pittsburgh (8)
    5 Baltimore (8)
    6 NewOrleans (8)
    7 TampaBay (7)
    8 GreenBay (7)
    9 Miami (6)
    10 Philadelphia (7)
    11 Chicago (8)
    12 Tennessee (5)
    13 NYGiants (7)
    14 Indianapolis (6)
    15 KansasCity (7)
    16 Cleveland (4)
    17 SanDiego (6)
    18 Jacksonville (6)
    19 Washington (5)
    20 Minnesota (4)
    21 Oakland (5)
    22 Seattle (5)
    23 Houston (5)
    24 StLouis (5)
    25 SanFrancisco (4)
    26 Denver (3)
    27 Dallas (3)
    28 Arizona (3)
    29 Buffalo (2)
    30 Cincinnati (2)
    31 Detroit (2)
    32 Carolina (1)

    Cheers.