Smart, or Overfit?

At the NBER conference, Jonathan Berk of UC Berkeley made a very spirited, but unconvincing, defense of the idea that one should evaluate skill, not through alpha via an asset pricing model (eg, the Fama French 4 factor model), but rather, look at the size of the fund and its fees as evidence of alpha. That is, if a firm has edge, in a competitive environment, its edge is equal to its fees. This argument is based on competitive markets, and makes some sense. But it implies that mutual fund returns, pre-fees and expenses, are correlated with fees and expenses. I haven't seen such data.

Anyway, there was a fun 15 minute argument about the issue. Berk is a very impressive guy, he looks, and acts, like an ultimate fighter: aggressive, shaved head. He has one of those English accents that's very distinctive, but as an American, I don't know where it comes from. I just know it's not like the Queen, nor a cockney accent. After that, who knows. I'm sure Englishmen could put him in a particular county and social strata.

Berk made an interesting observation. Any market anomaly that has been expertly exploited, is observationally equivalent to any overfit, ex post observation. Both are merely historical, and though you can come up with a good explanation for why making money on internet companies in the 90's after the fact, one could easily argue these were merely accidents, unforeseeable. The bottom line is, you really can never know, except through a very qualitative evaluation of the motivation.

There was a clear distinction between theorists and empiricists. Theorists merely want models. They want something with generalization, something they can extend or modify, or else, what are they to do? People want things they can use, and so, as an intellectual, this means an idea that is amenable to various refinements. There is an interesting contrast between the desire to make something useful pedagogically, or useful for refinements, and useful for explaining the data. Sometimes they coincide, sometimes they don't.