On this page
You paste a probability question into a chatbot. It says 671/1296. You press regenerate and it says 2/3. Nothing is broken: chatbots are built this way, and the second answer can become a tool for checking your work.
Why does AI give different answers to the same question?
A chatbot writes its reply one small piece at a time. A piece is called a token: a word or part of a word. At each step the model gives every possible next token a probability, then picks one at random, weighted by those probabilities. Researchers at Thinking Machines Lab describe this step as sampling: the model's output is turned into a probability distribution, and the system "probabilistically selects a token" (Thinking Machines Lab, 2025).
Temperature is the setting that controls how random a model's output is: higher temperatures give more random output, lower temperatures less (Google machine learning glossary).
One different word early on can send the whole reply down another path. Turning the randomness off does not make answers perfectly repeatable either. In a September 2025 test, Thinking Machines Lab sent one open model the same prompt 1,000 times at temperature 0 and got 80 different completions; all of them matched for the first 102 tokens and then split (Thinking Machines Lab). They traced the cause to how requests are batched together on the server, and with their fix all 1,000 completions came out identical.
What does it mean when a maths answer changes?
For an essay, two different replies can both be fine. For a maths problem with one correct answer, two different final answers mean at least one of them is wrong. That is a signal you can use.
Researchers have used this signal on purpose. Xuezhi Wang and colleagues had a language model write many separate lines of reasoning for the same problem, then took the answer that came up most often. They called the method self-consistency, and their abstract reports an improvement of +17.9% on the GSM8K maths word-problem benchmark over standard chain-of-thought prompting, which takes a single answer (Wang et al., ICLR 2023).
A second team, Potsawee Manakul, Adian Liusie and Mark Gales, built a checker called SelfCheckGPT on a related idea: when a model really knows something, repeated samples tend to agree, and when it is making things up, the samples tend to contradict each other (Manakul et al., EMNLP 2023). They tested it on biography passages, not maths, but the principle carries over.
How do you use repeat asks as a check?
Try the problem yourself first, even roughly, so you have your own answer to compare.
Open a new chat each time. In the same chat, the model can see its earlier reply and may simply repeat it.
Paste the exact same question three times and write down only the final answers.
Read the pattern using the rules below.
All three match, and match yours: a good sign. Still check one step by substituting back or estimating.
Two against one: find the first line where the odd one out differs, and work that line yourself.
All three differ: the question may be unclear or too hard for the chatbot. Go to your textbook, your teacher, or a tool that calculates exactly.
What does a repeat-ask check look like on a real problem?
Question: what is the probability of getting at least one six in four rolls of a fair die?
Say the three runs give 671/1296, 2/3 and 671/1296. Line them up and the split is in the very first step. The 2/3 run added the chances: 4 × 1/6. The other two found the chance of no six at all, (5/6)⁴ = 625/1296, and subtracted it from 1 to get 671/1296, about 0.52.
Now you decide, not the chatbot. Try the adding method on seven rolls: 7 × 1/6 = 7/6, a probability bigger than 1, which is impossible. Adding fails because the events overlap (you can roll two sixes), so 671/1296 is right. For more ways to test this kind of answer, see three ways to check an AI probability answer.
The repeat ask did not hand you the answer. It pointed you to the exact step worth thinking about, which is where the learning happens. If that step shows you picked the wrong method, the Fermi Problem Library has worked problems with a step-by-step solver that explains the reasoning rather than printing an answer.
When should you not rely on asking again?
Long exact calculations: a calculator or computer algebra tool is built for these. Here is when to trust computer algebra over a chatbot for maths.
Your answer key and the chatbot disagree: follow this five-step check for when the answer key and AI differ.
For parents, one question does most of the work: "What did the other run say, and which step is different?" It turns a confusing pair of answers into a short maths conversation.
We built Fermi's AI tutor to teach rather than just answer: it notices where a student is stuck and checks understanding as it goes.
