Simpson's Paradox

Last week, Paul Krugman argued that kids in Wisconsin outperform kids in Texas, which to him implied this was due to the collective bargaining rights of Wisconsin educators (Texas does not have those).

IowaHawk noted this finding isn't what it appears. As he notes:

white students in Texas perform better than white students in Wisconsin,black students in Texas perform better than black students in Wisconsin,Hispanic students in Texas perform better than Hispanic students in Wisconsin.

So, on average Wisconsin outperforms Texas, but within each ethnicity, Texas outperforms Wisconsin. As ethnic performance varies predictably across all states, ignoring the ethnic composition of states is really misleading, and is why my state, Minnesota, appears to do so well on these simple measures.

It's the old Simpson's Paradox! It's a really fun issue that pops up often, and good to be aware of because it's one of those cases where an omitted variables bias is at work. That is, the effect of x on y shows a significant coefficient, but when you include another variable, the coefficient on x goes to zero or even changes sign. This usually shows up in a simple cross-tabulation, and it's good practice to always be looking for other variables that could affect your findings, because if you are trying to make money like I am, it's good to know the truth (if your a partisan shill it's a minor annoyance).

Here's my application of Simpson's Paradox to the CAPM, something I go over in Finding Alpha (this is from Chapter 1 of my videos):


One could also adjust for price and also find this gets rid of the positive relation between beta and average returns. Much of the apparent relation between price or size and returns, meanwhile, is the result of measurement errors such as the delisting bias and the problems of geometric averaging (I know CRSP says they got rid of the delisting bias, but there's still a huge low-price premium in their data, and that's obviously not obtainable, as evidenced by the failure of low-priced funds that were popular in the late 80's--now gone).