10.4Base rates and risk

Should a positive test make you panic? A test result and a risk headline both hide the same missing number: how common the thing was to begin with. Set it below, count the people, and watch an accurate test turn out mostly wrong and a doubled risk turn out tiny.

Runs in your browser · settings are saved in the link · this page explains how the numbers work and is not medical advice

1Read
Your doctor calls: a routine screening test came back positive. Say it is a mammogram, a breast X-ray offered to women with no symptoms. Among women like you, 1% have breast cancer. The test finds 90% of cancers, and it wrongly flags 9% of women who do not have one. These are the figures Gerd Gigerenzer and colleagues use to teach doctors.
Before you go on, guess: given the positive result, what is the chance it is cancer? About 90%? About 80%? About 1 in 10? About 1%?
2Set
Here are 1,000 people who took the same screening test, one square each. Start with how common the condition is among people like them: the prevalence, or base rate. Pick one of the examples or set it yourself.
Load an example
People who have the condition: 1%
- Has the condition
- Does not
10 of 1,000 people have the condition. The other 990 do not.
3Test
A test has two accuracies. Sensitivity is how many of the sick it catches. Specificity is how many of the healthy it clears; the rest of the healthy get a false alarm. Set both, then run the test on all 1,000 people.
Sensitivity, sick people who test positive: 90%
Specificity, healthy people who test negative: 91%
- Has the condition
- Does not
Press Run the test to see who tests positive.
4Count
Now ignore everyone who tested negative and look only at the positives. That is where you are when the doctor calls.
Of 98 people who tested positive, 9 are sick.
Out of 1,000 people, rounded to whole people. 10 have the condition and 9 of them test positive; 990 do not, and 89 of them test positive anyway.
A positive is real
9%
A negative is really clear
99.8%
The test is right
91%
Run the test in step 3 to see these people in the field.
Note
A test that is 99% sensitive and 99% specific sounds as good as certain. If 1 in 100 people have the condition, only 50% of those who test positive are sick. If 1 in 1,000 have it, only 9% are. The test did not get worse. There were simply far more healthy people for it to make its 1% of mistakes on.
With the mammography numbers (1% have cancer, 90% sensitivity, 9% false alarms), 9% of positives are cancer, about 1 in 10. When Gerd Gigerenzer gave those numbers as percentages to 160 gynaecologists in 2007, most answered 90% or 81%, and 21% chose “about 1 in 10”. After learning to restate them as counts of women, as in this step, 87% did.
5Retest
Take only the 98 people who tested positive and test them again with a second test that is just as accurate and makes its mistakes independently. Among them the condition is no longer rare: it starts at 9%.
- Sick, positive
- Healthy, positive
Press Run the second test.
Positive is real, one test
9%
Positive twice is real
50%
Note
This is why a positive screening result is usually followed by a second, different test before anyone acts on it. The first test does not need to be decisive. Its job is to turn a rare condition into a common one among the people sent for the next test, where the same accuracy goes much further.
6Compare
Base rates matter for headlines too. “Raises your risk by 18%” is a relative change: 18% of whatever the risk was. Pick an example or set your own, then switch the headline between the relative number and the absolute one.
Load a headline
Risk to begin with: 39 in 1,000
Relative increase: 18%
Write the headline as
A daily 50 g portion of processed meat raises your risk of bowel cancer by 18%
- Cases anyway
- Extra cases
- No case
Out of 1,000 people, about 39 get bowel cancer anyway, and about 7 more get it with the exposure. That is one extra case for every 142 people exposed.
Before
39 in 1,000
After
46 in 1,000
People exposed per extra case
142
Note
In October 1995 the United Kingdom's Committee on Safety of Medicines warned that newer contraceptive pills doubled the risk of dangerous blood clots, a 100% increase. In absolute terms it was 1 more case in every 7,000 women: about 1 in 7,000 on the older pill, about 2 in 7,000 on the newer one. Many women stopped taking the pill, and Gigerenzer and colleagues report an estimated 13,000 extra abortions in England and Wales the following year.
A relative number on its own cannot tell you whether to worry. Ask for the base rate: out of how many, before and after?
7Go
Next time you meet a test result or a risk headline, ask two questions: how common was it to begin with, and out of how many? Then practise on these.

10.4.1How the model works

The chance a positive result is real is the positive predictive value. Out of everyone, a share equal to the prevalence is sick, and the test flags a share of them equal to its sensitivity. Of the healthy, it flags a share equal to one minus its specificity. The positive predictive value is the first group divided by both groups together: sensitivity × prevalence ÷ (sensitivity × prevalence + (1 − specificity) × (1 − prevalence)). That is Bayes' rule. The page works it as counts of people instead of percentages, because counts are what people reason with best. In Gigerenzer and Hoffrage's 1995 experiments, people with no training in statistics reasoned their way to the right answer in 46 to 50% of problems given as frequencies, against 16 to 28% of the same problems given as probabilities.

The counts are rounded to whole people, so they can differ from the percentage by a point. The second test assumes its mistakes are independent of the first's. Repeating the same test often breaks that, since whatever fooled it once can fool it again, which is one reason a confirming test is usually a different kind of test.

The risk panel is plain arithmetic: the risk after is the risk before times one plus the relative increase. The processed meat example applies the 18% from the International Agency for Research on Cancer (IARC) to the 3.9% lifetime risk of bowel cancer in the United States. That is a rough illustration, not an estimate for anyone: the 18% comes from studies of a daily 50 g portion, and the lifetime figure covers everybody, whatever they eat. Your own risk depends on things this page does not know, which is what a doctor is for.

10.4Base rates and risk

10.4.1How the model works

10.4.2Sources