Checking the numbers

Via an entry at Pandagon I see that John Lott (of smackdown by Tim Lambert fame) has gone to Rush Limbaugh’s aid by trying to statistically establish that sports media does, indeed, treat black quarterbacks more favorably than white QBs. Obvious questions aside—how does Lott assemble ten research assistants so quickly? Is there a ready pool of Lexus-Nexis dredgers on hand at the National Review? Is the quickly-assembled methodology of this project reliable and valid?—Lott’s attempt to give scientific legitimacy to the rantings of a blowhard is problematic. Why give benefit of the doubt to comments that are part of a long line of right-wing exaggerations and outright falsehoods?

Yet, because all I had to do this afternoon was grade sixty papers on the sick role and processes of medicalization, I thought what the heck and took a look at Lott’s quarterback data, using his stata output to replicate the results. The data is a record of quarterbacks’ wins, opponents, some relevant game statistics (points scored, points scored against), and race information, for each team for four weeks. The variable of interest is what Lott and his ten-man team calculated to be the “percent favorable news coverage” in a given week. My understanding of tobit regression—the tool used by Lott in this case—is a little hazy, and I’m not even sure it’s appropriate in this circumstance, so I give Lott the benefit of the doubt and keep on using tobit, with his specification. [1]

Step one: Sure enough, I can dump Lott’s data into Stata and make it work. Well, after correcting a variable name or two. Results look about the same: Black QB has a significant effect on positive news coverage (p<.031). Maybe Lott’s on to something (though the effect of quarterback rating has a much higher significance level).

Step two: Take a look at the data. Does coverage depend on lag of the team’s performance the previous week? I generate a quick dummy variable to check this, and find no changes in the results.

Step three: Does positive news coverage depend on whether the team has been picked to win the Super Bowl? I look up ESPN’s long-range picks and count up the number of commentators who pick a given team to win it all. (Rams: 1; Titans: 6; Patriots: 3; Chiefs: 1; Bucs: 6; Raiders: 1. Just so you know.) When I add these numbers to team records, I notice something interesting about the data: Some teams are listed more than once, because they have more than one active quarterback. This turns out to be important.

Step four: No effect of pre-season championship picks, at least not with the quick-and-dirty coding I gave it. So, I go back and look at those teams with multiple quarterbacks (there are four in the data set: the Jaguars, the Vikings, the Browns, and the Rams). Different QBs play for these teams sometimes from week to week, and I’m not sure what that does to an analysis of the quarterback’s news coverage. [2]

With this uncertainty, I drop all teams with more than one quarterback from the data set, and repeat the tobit analysis. Black QB immediately drops far below the level of statistical significance, leaving only—not surprisingly—quarterback rating to have a significant effect.

Something interesting happens with those teams that have more than one quarterback. When they’re removed, there is absolutely no effect of QB race on media coverage. So Step six: I replace those teams in the data and add control variables denoting whether teams a) have more than one QB, and b) have two QBs, one of which is black. Then I add an interaction term that captures when the black quarterback of a two-QB team plays in a given week.

And what happens: When Lott’s tobit model is repeated with this interaction term included, the effect of race vanishes.

So where does this put us? Heck, I don’t know. I don’t really follow football. But what the statistics suggest is that the effect of race on media coverage is highly dependent on the presence of only four teams who have more than one quarterback and that that effect of race disappears when we control for which quarterback plays in a given week.

[1] Just for fun, I ran straightforward regressions on the data, one week at a time, and found the variable for black QB to have a weakly significant and positive (p<.1) effect on positive news coverage in only one week.

[2] Frankly, I’m not even sure that tracking this data week by week is the right way to go; if the effect of race really is independent of the vagaries of weekly play, after all, perhaps each QB should have just a single entry, rather than one per week. Performing just such an aggregation and then modeling results in the variable for black QB coming in a very, very long way from statistical significance.