History
Famous blunders and their causes
The great historical mistakes share a shape: the warning existed, an alternative existed, and the people in charge had reasons not to hear it.
Barbara Tuchman, The March of Folly (1984), which sets tests for calling a policy a folly; the causes map onto the biases catalogued by Kahneman, Tversky and Munger.
Tuchman set three tests before she would call a decision a folly rather than a mistake. The policy had to be seen as counterproductive in its own time, not only in hindsight; a feasible alternative had to be available; and it had to be the policy of a group, carried on past any one leader. Her cases were the Trojan horse, the Renaissance popes who provoked the Reformation, the British ministers who lost the American colonies and the United States in Vietnam. As a model, the pattern predicts something you can look for in any disaster: a warning on the record before the event, from people with the facts, which the decision-makers knew about and overrode. Reading a blunder, then, means finding the warning and asking why it lost.
The Challenger launch shows the recurring causes together. Booster O-ring erosion was first seen on the second Shuttle flight in November 1981; as later flights confirmed the damage, the Rogers Commission found, NASA and Thiokol responded by raising the amount of damage they considered acceptable, which is confirmation bias. A system the Commission called strained by nine flights in 1985 had 15 scheduled for 1986, and a launch constraint imposed in mid-1985 was waived for every flight after it: schedule as sunk cost. The contractor whose engineers objected depended on the customer who wanted to fly, and its managers were told to think like managers rather than engineers, which is incentive-caused bias. And the objection never travelled up to the people who gave the final go. Each cause is ordinary alone; together, under a deadline, they are how a known risk gets taken.
The trap in reading history this way is hindsight. Baruch Fischhoff showed in 1975 that people told how an event turned out rate that outcome as having been more likely all along, and do not notice the knowledge working on them. Gordon Wood made the same objection to Tuchman: whether an action ends up as an example of Churchillian determination or of wooden-headedness, he wrote, seems to depend on the unpredictable outcome of events. Her first test is the defence. If you cannot find a contemporaneous warning from someone who had the facts, you are probably looking at a bet that lost, not a folly, and the ignored warnings that turned out fine never make it into the books. Hold the standard, or every failure becomes someone's folly.
Example
On the evening of 27 January 1986, in a teleconference that began at about 8:45 pm, engineers at Morton Thiokol recommended that Challenger not launch below 53 degrees Fahrenheit, the coldest temperature at which the solid rocket boosters had flown; the forecast for the 9:38 am launch was in the low twenties to about 26 degrees. After NASA (National Aeronautics and Space Administration) managers pushed back, Thiokol's Jerry Mason told Bob Lund to take off his engineering hat and put on his management hat, and the company reversed its recommendation. Challenger launched the next morning at 36 degrees, 15 degrees colder than any previous flight, and broke up 73 seconds later after the aft field joint of the right booster failed. The Rogers Commission reported on 6 June 1986 that the decision process had a serious flaw and that the engineers' argument never reached the managers who gave the final go.
Where it applies
In an incident review, the question is not who erred but which warning existed and why it lost; the answer is usually a deadline, a contract or a channel that did not reach the decider. Before a model launch, write down the evaluation that would make you stop, and who has the standing to call it, because that is exactly what Thiokol's engineers lacked.
Think about it
Which warning in your current project has been raised, acknowledged and quietly dropped, and what would it take for it to reach the person who decides?
Applied it to something real? Log it.
Test yourselfWhat three tests did Tuchman set before calling a decision a folly, and which one protects you from hindsight?
Contemporaries saw it as counterproductive, a feasible alternative existed, and a group rather than one ruler pursued it. The first test needs a warning on the record at the time, not a judgement made after the outcome.
Related models
- Confirmation biasPeople look for, notice and remember evidence that fits what they already believe, and rarely test what would prove them wrong.
- The sunk cost fallacyMoney, time or effort already spent cannot be recovered, so it should not count in what you do next; people count it anyway.
- Incentive-caused biasAn incentive bends your own judgement towards the conclusion that pays, and you will not feel it happening.
- Pre-mortemBefore you start, imagine the project has already failed and ask everyone to explain why.
- Survivorship biasWhen you only see the cases that passed a filter, the pattern in them is partly a pattern of the filter.
Sources and further reading
- Rogers Commission, Report of the Presidential Commission on the Space Shuttle Challenger Accident, chapter V, The Contributing Cause of the Accident (1986)
- Rogers Commission, Report of the Presidential Commission on the Space Shuttle Challenger Accident, chapter IV, The Cause of the Accident (1986)
- Gordon S. Wood, History Lessons, review of The March of Folly, New York Review of Books (29 March 1984)