Base Rate Fallacy

In plain terms

Every piece of evidence lands on top of something you already knew: how common the thing is to begin with. That background frequency is the base rate. Skip it and even a very accurate test will point you the wrong way.

The reason is arithmetic, not intuition. When something is rare, the pool of people who don't have it is enormous. A small error rate applied to an enormous pool produces more mistakes than the true cases produce hits.

So "the test is 99% accurate and it came back positive" is not the same claim as "there's a 99% chance I have it." The second number depends on how rare the thing is, and you can't work it out until you look that up.

Why it matters

The question people answer is "how accurate is the test?" The question that decides the case is "out of everyone who tests positive, what share actually has it?" Those are different numbers, and for a rare condition they can differ by a factor of a hundred.

The shape repeats well outside medicine. Fraud screens, security flags, plagiarism detectors, spam filters, database DNA matches: each hunts for something rare in a very large haystack, and each one generates false alarms that outnumber the real hits. Treating the alarm as near-proof is the error.

None of this makes the test useless. A positive result usually does move the odds, sometimes enormously. It moves them from a starting point most people never bother to find.

Canonical example

"The screening test is 99% accurate and mine came back positive. So I almost certainly have it."

Take a disease that affects 1 in 10,000 people and a test that is 99% accurate in both directions: it catches 99% of real cases, and it wrongly flags 1% of healthy people. Screen a million people.

• 100 of them have the disease. The test flags 99 of those.
• 999,900 do not. The test wrongly flags 1% of them, which is 9,999 people.

That's 10,098 positive results, of which 99 are real. Under 1%. Fewer than one positive in a hundred is a true case, from a test that is right 99% of the time.

The positive result did do something. It moved the odds from 1 in 10,000 to about 1 in 102, close to a hundredfold shift. It just isn't a diagnosis, which is exactly why screening programs confirm with a second, different test rather than acting on the first.

Counter-example (not a fallacy)

"We only run this test on patients who already have the symptom triad. In that group, roughly 1 in 20 turns out to have the disease. A positive result here is worth acting on."

Same test, different starting population, different conclusion. Out of 2,000 such patients, 100 have the disease and the test flags 99 of them. The other 1,900 produce 19 false positives. Now 99 of 118 positives are real, about 84%.

Nothing about the test changed. The base rate changed, from 1 in 10,000 to 1 in 20, and it carried the answer with it. A doctor who orders the test only after the symptoms appear is using the base rate correctly, not ignoring it.

The line: is the background frequency part of the reasoning, or missing from it? Reaching a confident conclusion isn't the error. Reaching one without ever asking how common the thing is, is.

How to fix it

If you've been linked here, the fix is one extra number. Before you interpret a hit, find out how often the thing you're testing for occurs in the population you're testing. Then run the four-box count: true positives, false positives, true negatives, false negatives. It takes a minute on the back of an envelope and it usually changes the answer. If you can't find the base rate, say so and soften the claim, because without it "the test is 99% accurate" simply does not support "I've almost certainly got it."

If you're on the receiving end, don't argue with the accuracy figure. Ask the other question: "How common is this in the group being tested?" That reframes the discussion from the quality of the test to the thing that actually determines what a positive result means.