Updating a belief on new evidence

Imagine a disease that one person in a thousand has. There is a test for it. The test is right 99 times in 100. You take it. It says yes. How worried should you be?

Most people say 99%. The right answer is about 9%. Here is why, with no algebra at all.

Picture 1,000 people like you. One of them has the disease, and the test almost certainly says yes for that person. The other 999 do not have it. The test is wrong one time in a hundred, so it says yes for about ten of them anyway. Now count the people who got a yes. There are eleven. Only one of them is ill. One in eleven is about 9%.

The number that surprises people is the 999. It is easy to think about how good the test is and forget how rare the disease is. The rare thing stays rare even after a yes. It is just less rare than before: you went from one in a thousand to one in eleven. That is a big move. It is not a verdict.

This is why a second test helps so much. You start the second test believing one in eleven rather than one in a thousand. Another yes now leaves you near 90%. The evidence did not change. What changed is what you brought to it.

The same counting works anywhere. A warning light that goes off often tells you little when the fault is rare. A strong result in a field where most ideas are wrong is weaker than it looks. Ask two questions in order: how common is this to begin with, and how often does the evidence show up when it is not true.