Skip to content
Rabin Poudyal
Page 6-1
6-1 Models

6.33Fail-safe design

Systems and complexity

Every part will fail one day, so choose in advance which way it fails, and make sure that way does less harm than the thing it was guarding against.

Railway and machine engineering: George Westinghouse's 1872 air brake patent applies a car's brakes automatically if it comes loose from the train. Jerome Saltzer and Michael Schroeder made fail-safe defaults a principle of computer security in The Protection of Information in Computer Systems, Proceedings of the Institute of Electrical and Electronics Engineers (IEEE), 1975.

  1. 1Read

    A fail-safe design picks the state a part falls into when it breaks, and makes that state the safe one. In Westinghouse's patent a port opens when a car becomes disconnected from the train, and air stored on the car applies its brakes. Saltzer and Schroeder carried the idea into software: base access decisions on permission rather than exclusion. A mistake in a mechanism that grants explicit permission tends to fail by refusing access, "a safe situation, since it will be quickly detected"; a mistake in one that lists exclusions tends to fail by letting people in, which may go unnoticed. In software the two directions are usually called failing closed (refuse, stop, block) and failing open (let it through, carry on). A design whose failure state is the harmful one is sometimes called fail-deadly.

    The hard part is that safe depends on what you are protecting. A login check should fail closed; a fire exit should open when the power goes. A security filter that fails closed takes the service down with it, and one that fails open lets the attack through. The common mistake is choosing by accident: an unhandled error that ends the process is a decision to fail closed for every request, made by whoever wrote the error handling, often years earlier. Scale changes the answer too. A health check that removes a server when it looks sick fails safe for one server, and removes every server when they all look sick for the same reason at once.

    Failing safe is not free. A check that fails closed too readily causes outages and false alarms, and people learn to switch it off. There is often a third choice better than open or closed: keep the last known good version, or drop one feature and keep the rest. Decide the failure direction for each part on purpose, write it down next to the code, and test it by actually feeding the part missing, malformed and late input.

  2. 2Note

    Example

    On 18 November 2025 one bad configuration file reached Cloudflare's network and met its two proxies, which failed in opposite directions. A permissions change made a database query return every column twice, and the Bot Management feature file built from it crossed a limit of 200 features. In FL2, the newer proxy, the loading code panicked and requests failed with HTTP 5xx errors: it failed closed for all traffic. FL, the older proxy, raised no error and kept serving, but gave every request a bot score of zero, so customers with rules that block bots saw large numbers of false positives. Neither behaviour had been chosen for this case. Cloudflare's list of fixes included treating its own generated configuration files the way it treats user input, more global kill switches, and reviewing failure modes for error conditions across all its core proxy modules.

  3. 3Apply

    Where it applies

    For every check, limit, filter and health probe, write down what it does when its own input is missing, malformed or late, and whether anyone chose that. A configuration loader should usually keep the last good file; an authorisation check should refuse. In a system built on a language model, decide what happens when the moderation call times out: does the request go through unchecked, or does the user get an error? Either can be right, but it should be a decision.

  4. 4Think

    Think about it

    Pick one safety check in a system you run: what does it do when the thing it reads is missing or malformed, and who decided that?

  5. 5Log

    Applied it to something real? Log it.

  6. 6Check
    Test yourselfSaltzer and Schroeder said to base access decisions on permission rather than exclusion. Why does that make failures safer?

    A bug in a mechanism that grants explicit permission tends to refuse access, which people notice quickly. A bug in one that lists exclusions tends to let people in, which can go unnoticed.

  7. 7Go

    Related models

    • Margin of safetypage 6-53

      Leave a wide gap between what you expect to need and what you build or pay, so that your errors and bad luck do not sink you.

    • Postel's lawpage 6-64

      Be conservative in what you send and liberal in what you accept: it got the early internet to interoperate, and it lets errors harden into the standard.

    • Redundancy and slackpage 6-69

      Spare parts and spare capacity look like waste until the day they are the only thing between a fault and a failure.

    • Unintended consequencespage 6-108

      Purposeful actions produce outcomes nobody planned, because the people and systems acted on respond in ways the plan never modelled.

  8. 8See

    Seen in an incident

  9. 9Source

    Sources and further reading