Probability and statistics
Base rates
How common something is to begin with matters as much as the evidence in front of you, and people routinely leave it out.
Named the base-rate fallacy by Maya Bar-Hillel (1980), building on Kahneman and Tversky's 1973 work on representativeness; Meehl and Rosen had warned clinicians about it in 1955.
A screening test flags a rare condition, the test is 99 percent accurate, and whoever holds the result treats it as near certain. That is base rate neglect. The 99 percent describes the test; what you want is the chance that this positive is real, and that depends on how rare the condition is. Take 1,000,000 people and a condition affecting 1 in 10,000. One hundred have it, and a test with 99 percent sensitivity catches 99 of them. The other 999,900 are healthy, and a test with 99 percent specificity wrongly flags 1 percent of them: 9,999 people. So 10,098 people get a positive result and 99 of them are ill, just under 1 percent. Bar-Hillel's 1980 paper collected the experiments showing that people answer such questions from the specific evidence and drop the rate.
Bar-Hillel's explanation is relevance. People rank information by how specifically it seems to describe the case in front of them: a witness's testimony or a test result feels like it is about this person, while a population rate feels like it is about nobody in particular, so the rate loses. The fix that works best in experiments is to change the format. Gigerenzer and Hoffrage showed in 1995 that stating the same problem as counts ('of 1,000 people, 40 are infected and 30 of those test positive; 120 of the healthy also test positive') lets most people find the answer the percentage version hides. The 2002 Cognition paper by Hoffrage, Gigerenzer, Krauss and Martignon reviews that work: with natural frequencies the posterior is just 30 divided by 30 plus 120.
Base rates are not a trump card. The right reference class is a judgement: the rate among people who walked into a clinic with symptoms is not the rate in the general population, and when the evidence is strong enough the rate matters less. In the 1960s psychologists believed people over-weighted base rates (the finding was called conservatism), and the two literatures were never fully reconciled, so the honest claim is that people combine the two badly in both directions depending on format and framing. The practical rule survives: before believing a test, a detector or an interview impression, ask how common the thing was before you looked.
Example
In her 1980 paper in Acta Psychologica, Maya Bar-Hillel used the cab problem that Kahneman and Tversky had posed in 1972. A city's cabs are 85 percent Blue and 15 percent Green; a witness to a night-time hit-and-run says the cab was Green, and the court finds the witness identifies colours correctly 80 percent of the time. The witness evidence multiplies the prior odds of 15 to 85 by 80 to 20, which gives 12 to 17, so the cab was Green with probability 41 percent. The typical answer was 80 percent, because people used only the witness and dropped the base rate.
Where it applies
An anomaly detector with a 1 percent false positive rate on a million events a day raises 10,000 false alarms before it finds a single real intrusion if intrusions are rare, so the on-call rota is sized by the base rate, not by the detector's accuracy. The same arithmetic governs a hiring screen, a fraud flag and a large language model (LLM) classifier run over all your traffic.
Think about it
Which number did you treat as near certain this week because the evidence felt specific, without asking how common the thing was to begin with?
Applied it to something real? Log it.
Test yourselfA condition affects 1 in 10,000 people and a test is 99 percent sensitive and 99 percent specific. Roughly what share of positive results are real?
About 1 percent. Per million people there are 99 true positives against 9,999 false ones, because 1 percent of 999,900 healthy people is far more than 100 ill people.
Related models
- Bayes' ruleUpdate a belief by multiplying what you thought before by how much better the evidence fits one explanation than the other.
- The availability heuristicWe judge how likely something is by how easily examples come to mind, so vivid and recent things look more common than they are.
- Probabilistic thinkingThink in odds rather than certainties: estimate how likely each outcome is, and judge a decision by those odds rather than by the single result that happened.
- Selection biasIf the way cases got into your sample is related to what you are measuring, the sample will mislead you however large it is.
- The planning fallacyPeople predict their own projects will finish sooner and cost less than almost every comparable past project did.