When Theory Fails

Horgan and Johnson have a neat dialogue, as usual, over at Blogging Heads.tv. Horgan notes the Tetlock's book documented that experts did worse than chance at predicting major events, relative to a naive alternative (discussed in this good New Yorker here). The idea is that experts are incented to create innovative theories, and the more they know, they better they become at defending, and cherry picking the evidence to support it.


Another example of this is an experiment discussed in the video clip above, where rats were presented with a maze where they were presented food 60% of the time on the left. Quickly, the rats would always go left, successful 60% of the time (smart rats). In contrast, Yale students were presented with the same puzzle but were only successful 52% of the time because they were trying to guess complex patterns. Theory can be an impediment when you infer theories that aren't there.