On this page
Imagine a test that is 99 percent accurate comes back positive. It is tempting to read that as a 99 percent chance of disease. If the disease is rare, the real chance can be closer to 1 in 6. To see why, stop thinking in percentages and count people.
What does a 99 percent accurate test actually mean?
"Accuracy" usually hides two separate numbers. Sensitivity is the probability that a test correctly identifies people who truly have a condition, and specificity is the probability that it correctly identifies people who truly do not, as defined in a 2017 review in Frontiers in Public Health.
After a positive result, you need a third number. Positive predictive value is the probability that a person with a positive result really has the condition, and the same review notes that it depends on how common the condition is in the group tested.
In this post, 99 percent accurate means both 99 percent sensitivity and 99 percent specificity.
Why can a positive result still mean low odds of disease?
Picture 1,000 people as the 1,000 dots at the top of this page. Suppose the disease affects 2 in every 1,000 people (0.2 percent). Now test everyone:
Ill people: 2. The test catches 99 percent of them, so both test positive (1.98, rounded to 2).
Healthy people: 998. The test wrongly flags 1 percent of them, so about 10 test positive (9.98).
Total positives: about 12. Only 2 of them are ill.
So a positive means about 2 chances in 12, roughly 17 percent (the positive predictive value). Bayes' theorem gives the same answer: (0.99 × 0.002) ÷ (0.99 × 0.002 + 0.01 × 0.998) = 0.00198 ÷ 0.01196, about 16.6 percent. If the formula is new to you, our guide to Bayes' theorem and conditional probability builds it step by step.
Do doctors get this wrong too?
In a survey published in JAMA Internal Medicine in 2014, 61 physicians, house officers and medical students at a Boston-area hospital were asked about a disease with a prevalence of 1 in 1,000 and a test with a 5 percent false positive rate. The correct answer was about 2 percent; only 14 of the 61 (23 percent) got it, and the most common answer was 95 percent.
The format of the numbers matters. Gerd Gigerenzer and Ulrich Hoffrage reported in Psychological Review in 1995 that when Bayesian problems were given in frequency formats (counts such as 10 out of 1,000), statistically naive participants derived up to 50 percent of their inferences by Bayesian reasoning. A 2007 review by Gigerenzer and colleagues recommends natural frequencies instead of conditional probabilities when communicating health risks.
What changes when the disease is more common?
Predict first: if 1 in 10 people had the disease, would a positive mean more or less? Here is the same 99 percent test on 1,000 people:
1 ill: about 1 true positive and 10 false positives, so about 9 percent.
2 ill: about 2 true and 10 false, so about 17 percent.
10 ill (1 percent): about 10 true and 10 false, so 50 percent.
100 ill (10 percent): about 99 true and 9 false, so about 92 percent.
The test never changed; only the group did. As the Frontiers review stresses, predictive values depend on prevalence. This is one reason a positive screening result is often followed by a confirmatory test: the person now belongs to a small group where the disease is far more common.
How can you work out any test result in four steps?
This natural-frequency method, which Gigerenzer and colleagues recommend, works for any percentage in a news story, exam question or lab report:
Start with 1,000 people.
Use the prevalence to split them into ill and healthy.
Multiply the ill by the sensitivity (true positives), and the healthy by 1 minus the specificity (false positives).
Divide true positives by all positives.
Try it: a condition affects 5 in 1,000 people, and a test has 90 percent sensitivity and 95 percent specificity. What is the chance a positive is real? (Answer: about 4.5 true and 50 false positives, so roughly 8 percent.) For more traps, see our list of common probability mistakes.
If someone in your family gets an unexpected positive, ask the doctor:
How common is this condition in someone of my age and history?
How often does this test give a false positive?
Which test would confirm or rule it out?
If pictures like this grid help you, the Fermi Simulation Lab has interactive simulations for the ideas that are easier to see than to read, and the Fermi Concept Library gives every mathematics concept its own lesson.
