Stress Test Precision

It is important to have realistic objectives. Thus it was discomforting to know that in the words of White House Press Secretary Robert Gibbs

The stress tests were designed to ensure the administration “would have an exact diagnosis of what the problem was"

Exact? As there is considerable uncertainty as to the value of untraded bank assets, assessing their change in value given unemployment moving up to 13%, or housing assets fall 15%, is like asking what happens to Yankees if they lose their starting pitcher. Sure, they will be worse. How much worse? People will disagree. Indeed, the more knowledgeable the person, the greater the range in their disagreement. An ignorant person would probably just compare the wins of the pitcher to the average of the staff and take the difference. A more sophisticated Sabermetrics approach would look at the pitcher's recent performance, especially in spring season, how this compared to last year, as well as his best year.

Forecasting an exact diagnosis suggests the White House is clueless and expects to use this result as something that will seemingly clearly support whatever policy is in the pipeline (because the diagnosis is exact, the implication will be obvious). Their only problem is convincing people their stress tests are exact, which I think they will find impossible. Pointing to 'models' and sophisticated reasoning are not enough. Show us the assumptions, the simulations, the delineation of risk within the institution by obligor and facility, and there is no way you can not reject the hypothesis the number you generate is off by a factor of 3, which would generate a totally different policy recommendation.

Perfection is the enemy of the good, and by stating you expect a diagnosis to be perfect, you are ensuring failure. This is because if you believe bank assets and their sensitivities do not have large standard errors, you would do a different policy than if they do have large standard errors.

Consider crime. A realistic objective tries to minimize violence, and minimize the chance an innocent person will be convicted. An idealist will have try to eliminate both crime and wrongful convictions. The realistic approach will be more successful on type 1 and type 2 errors because it accepts its limitations.

I was watching Bloggingheads between two political scientist types, and they were talking about how 'complex' the problem is because you can't let ships carry small arms, you can't shoot all the pirates, the pirates are just trying to make money and generally don't hurt their victims (I snipped 1 minute of their whiny reasoning below). Fundamentally, there is no perfect solution, which in the end they noted was really irsksome. In their academic circles, this sounds like reasoned debate, but it is merely wimpy indecision in the face of a dilemma driven by an imperfect solution. It reminded me of the scene in Godfather Part 1 where the Don slaps a crying Johnny Fontaine and says 'you can act like a man!' Accept the solution will be imperfect, people will die, and some people will object.