Dishonest Forecasters

"Econophysicist Accurately Forecasts Gold Price Collapse" says the MIT Technology review. Didier Sornette seems to generate a lot of buzz, because the whole idea that something called 'complex-systems theory' can predict prices seems like a neat trick. His website notes a lot of press coverage, including Nature and the Wall Street Journal. Wikipedia states that 'complex systems theory' is "used as a broad term encompassing a research approach to problems in many diverse disciplines including anthropology, artificial intelligence, artificial life, chemistry, computer science, economics, evolutionary computation, earthquake prediction, meteorology, molecular biology, neuroscience, physics, psychology and sociology." A theory of everything indeed.

I find these models generally look like the multiplier-accelerator model that Paul A. Samuelson introduced back in 1939 because it seemed possible of mimicking the boom and bust nature of economic time series. That thread died because it was barren, but it was born of the same intuition: it looked like it could explain the future, because you could fit it to the past.

Econophysicists have always been on the cusp of some fantastic breakthrough, but these researchers seem to have a very selective memories. Sornette's December 2002 call for a stock market crash in 2003-4, could not have been more wrong, and it seems to have been put down the memory hole, nowhere on his website. Anyone who's even a little dishonest is not to be taken seriously, because once you start selectively picking from your prior forecasts, you generally don't stop.

A professor forwarded me some econophysics paper he co-authored that he thought was a paradigm shifter. It was basically some model that generated chaos, with some positive feedback loops, and I asked about some technical portion of it, he replied he didn't know about that, but his physics co-author did. This suggested to me the mechanism itself was not important, but rather, it generated the right big-picture pattern. How naive. If I want to generate a model of the stock market, I could rationalize a model of geometric brownian motion, with some Poisson jump process, perhaps some auto-regressive volatility, and say that looks like the S&P500, but that's hardly a model, just a description. It doesn't predict anymore than the rand() function predicts lottery numbers (they look like lottery numbers--very random!).

I remember the Stock-Watson model that was to replace the old Leading Economic Indicators in predicting recessions. Prior to the 1990 recession, it was generating a lot of buzz, because it used a Kalman filter, just like real rocket scientists, and seemed obviously better than the simple sum of ten correlates with business cycles. Yet the model failed to predict the 1990-1991 recession, and an updated version of the model then failed to predict the 2001 recession. Each time, however, Stock and Watson were very honest and clear about the degree it failed and why (see here). It's rather straightforward what they did, but remarkably rare for such prominent forecasters. Further, reading this research, I feel I learned something, which I never have done from the econophysicists (other than they don't realize that perfect competition, or Gaussian distributions are not necessary assumptions to most economic theories, see here).