On this page
An AI chatbot writes probability answers confidently, which makes a wrong one hard to spot. A question that looks like a famous puzzle often gets the famous puzzle's answer, even when the wording has changed. You do not need to redo the whole solution to catch this: three small checks will tell you whether to trust the number.
Why do AI chatbots get probability questions wrong?
In the GSM-Symbolic study by Mirzadeh and colleagues, adding a single clause that seemed relevant but did not change the maths caused accuracy drops of up to 65% across the state-of-the-art models tested.
A 2025 paper, Frontier LLMs Still Struggle with Simple Reasoning Tasks by Malek and colleagues, built "unpuzzles": famous puzzles rewritten so they become easy. Models that solved the originals tended to fail the easy versions, in patterns the authors link to memorising the originals.
Probability is full of famous puzzles. For the wider picture, see why AI chatbots get maths wrong.
How do you check an AI's probability answer?
Take one question: what is the chance of at least one six in four rolls of a die? Suppose the AI says 4 × 1/6 = 2/3.
Shrink and list. Cut the problem down until you can write out every outcome. With two rolls there are 36 outcomes, and 11 of them contain a six, so the answer is 11/36. The AI's method would give 2 × 1/6 = 12/36. It counted (6, 6) twice, so the method is wrong.
Flip to the complement. "At least one" is the opposite of "none", so P(at least one six) = 1 − P(no six) = 1 − (5/6)^4 = 671/1296, about 0.52. Then test the AI's formula on a bigger case: n/6 gives 7/6 for seven rolls, and no probability can be more than 1.
Simulate. Roll a real die four times, note any six, and repeat 50 times. Or use a spreadsheet: put =RANDBETWEEN(1,6) in four columns, copy it down 1,000 rows, and count the rows that contain a 6. With 1,000 rows you should land near 0.52, well below 0.67.
Where have AI models got probability wrong?
Case 1: Linda, with the trap removed. In the classic Linda problem, "bank teller" beats "bank teller and feminist", because adding a condition cannot make an event more likely. In the easy version from Malek and colleagues' paper, the question itself says Linda is active in the feminist movement, then asks: is she more likely a bank teller, or active in the feminist movement? The paper reports that o1 still picked "bank teller", repeating the original puzzle's reasoning.
Check with a list. Picture ten women who match the description: all ten are feminists, because the question says so, and at most ten are bank tellers. "Feminist" cannot lose.
Case 2: the birthday problem, flipped. Misguided Attention, a public collection of trick prompts for testing language models, asks: "In a room of 30 people, what's the probability that at least two do not share a birthday?" Its notes say most models give the solution to the original birthday problem instead. That original answer, the chance that at least two people share a birthday, is 70.6% for 30 people.
Use the complement. The opposite of "at least two do not share" is "all 30 share one birthday". Ignoring leap years, that has probability (1/365)^29, a number with more than 70 zeros after the decimal point. So the answer is practically certain. Shrink it: with just two people, the chance they differ is already 364/365.
Case 3: Monty Hall, inverted. The same Misguided Attention collection has an inverse Monty Hall: one door hides a donkey, two hide cars, you pick one, and the host opens another door to show a car. What should you do to win a car? Its notes say most models give a strategy that wins the donkey.
List the cases, assuming the host always reveals a car. You first pick the donkey 1 time in 3, and switching gets you a car. You first pick a car 2 times in 3, and switching gets you the donkey. So staying wins a car 2 times in 3. To simulate, use three playing cards (two red, one black) with a friend as host.
What should you do when the AI and your check disagree?
"Are you sure?" tells the AI nothing about where the error is. Give it your evidence instead:
"I listed the two-roll case and got 11/36. Your formula gives 12/36. Which step double-counts?"
"My 50 dice trials came out near half, not two-thirds. Explain the gap."
If the method keeps going wrong, the Fermi Problem Library has worked problems with a step-by-step solver that explains the reasoning rather than printing an answer. Fermi's AI tutor notices where you are stuck and decides whether to explain, ask or show, so you rebuild the method instead of copying a number. For exam-style traps, read five probability reasoning mistakes in JEE maths.
Next time an AI hands you a probability, pick one check and run it before you write the answer down.
