On this page
You ask a chatbot for the answer to a physics numerical, then type, "Are you sure?" It replies, "Yes, I'm 95% confident." That feels like a guarantee. It is not, and five quick checks tell you far more.
Does AI know when it is wrong?
Sometimes, on the inside, but it often does not tell you. A 2022 study by Anthropic researchers, Language Models (Mostly) Know What They Know, found that larger models are well calibrated on multiple-choice and true/false questions when given in the right format. Calibrated means that when a model gives an answer a 70% chance of being right, answers like that are right about 70% of the time.
What a model holds internally is not the same as what it writes in a chat. A 2025 paper by Adam Tauman Kalai and colleagues, Why Language Models Hallucinate, argues that the way models are trained and tested rewards a confident guess over "I don't know", much like a student who guesses every question on a paper with no negative marking.
Asking the model to recheck itself does not fix this. Jie Huang and colleagues (ICLR 2024) found that models struggle to correct their own reasoning without outside feedback, and sometimes get worse.
Why is "I'm 90% confident" weak evidence?
Because the number is written the way people write, not measured. In an ICLR 2024 study by Miao Xiong and colleagues, large language models that stated their confidence in words tended to be overconfident. Those stated confidence levels mostly fell between 80% and 100%, often in multiples of 5, and the authors suggest the models may be imitating how humans talk about confidence.
OpenAI's SimpleQA paper asked models for their best guess plus a confidence percentage and found that they consistently overstate it.
The other half of the problem is us. In a 2025 study in Nature Machine Intelligence, Mark Steyvers and colleagues found that people tend to overestimate how accurate AI answers are when they see the default explanations. Longer explanations raised people's confidence even when the extra length did not make the answers more accurate.
What signals can you check yourself?
Ask again in a fresh chat. In the Xiong study, checking whether 5 sampled answers from GPT-3.5 agreed flagged wrong answers to GSM8K maths word problems far better than a single answer with its stated confidence: the detection score (AUROC) rose from 54.8%, close to random guessing, to 92.7%. The SimpleQA paper saw the same pattern: across 100 tries, the more often a model gave the same answer, the more likely that answer was correct. If fresh answers disagree, trust none of them yet.
Plug the answer back in. Substitute it into the original equation, check the units, try a limiting case. Our guide to plugging the answer back in to check AI shows how.
Estimate first. A rough mental estimate catches answers that are off by a factor of ten. See why you should estimate before trusting an AI number.
Push back with a wrong answer. Suggest a wrong value and see whether it agrees. If it does, its earlier confidence meant little; read why AI sometimes agrees with your wrong answer.
Match a trusted source. For facts and exam rules, check your NCERT textbook, your teacher or the official exam website.
A worked example: checking a projectile answer
Say you ask: "A ball is thrown at 20 m/s at 30 degrees to the horizontal. Find its range. Take g = 10 m/s²." Imagine the chatbot answers 20 m and adds that it is 95% confident. Ignore the 95% and run the checks.
Estimate: the largest possible range at this speed is u²/g = 400/10 = 40 m, at 45 degrees. At 30 degrees, expect something in the 30s, not 20.
Formula: R = u² sin 2θ / g = 400 × sin 60° / 10, which is about 34.6 m.
Ask again: paste the same question into a new chat. If it now says 34.6 m, the first answer was a slip, whatever confidence came with it.
Then try this script: "Show each step with units. Substitute your answer back in, and tell me which step you are least sure of." A step you can check is worth more than any percentage.
How do you make checking a habit?
Pick one check per subject: estimation and units for physics, plugging back in for maths, the textbook for biology and chemistry facts. For reasoning you can follow instead of a confidence number, the Fermi Problem Library has worked problems with a step-by-step solver that explains the reasoning rather than printing an answer. And Fermi's AI tutor works through problems alongside you and checks understanding as it goes, so checking becomes part of the lesson.
Next time a chatbot says it is sure, ask it to show you why.
