On this page
When you are studying for a physics or mathematics exam, an AI chatbot can seem like an infinitely patient tutor. However, an AI is not a calculator; it is a language model predicting the most likely next word based on patterns in its training data. This means it can confidently invent false logic, especially when a problem requires building a robust mental model of the physical world.
A 2024 study on college-level engineering physics found that while GPT-4 solved 62.5% of "well-specified" textbook problems, its accuracy dropped to just 8.3% for "under-specified" real-world problems where some data was missing. When the details are messy, AI often fails to make reasonable assumptions.
Because of this error rate, you cannot read an AI's response passively. To avoid learning the wrong method or memorising a hallucinated fact, run every AI response through this four-point checklist.
Does it answer my question?
Often, you ask an AI a very specific question, such as "Why is the gravitational potential energy negative in this step?" Instead of answering that exact doubt, the AI might reply with a five-paragraph summary of gravity, Newton's laws and a generic example.
This happens because the model associates your keywords with broad textbook explanations. If you read the long response passively, you might feel you have learned something, but your original doubt remains unresolved. Before reading the entire wall of text, scan the first sentence. If it does not address your specific "why" or "how", prompt it again: "You gave me a summary, but you didn't answer my specific question about the negative sign. Please explain only that."
If you need a system that focuses on your actual gaps rather than generic summaries, Fermi's AI tutor teaches rather than answers: it holds a line of reasoning across a whole session, notices where a student is stuck, and decides whether to explain, ask, or show.
Does it use the method from my syllabus?
Different curricula and exam boards require different approaches. For example, your board exam marking scheme might demand you solve a quadratic equation by completing the square, but the AI might use the quadratic formula because it is statistically more common in its training data. Alternatively, it might use university-level calculus to solve a kinematics problem that your Class 11 textbook solves using standard algebraic equations of motion.
Always compare the AI's method against your textbook or class notes. If it introduces a theorem, a shortcut or a formula you have never seen in class, do not use it. You will likely lose marks in an exam for applying out-of-syllabus methods without showing the required intermediate steps.
For guidance on matching AI methods to your specific curriculum, read our guide on why AI chatbots get maths wrong, and a five-step way to check.
Are the units consistent at every step?
One of the most common failure modes in AI problem-solving is dropping units halfway through a calculation and confidently sticking the desired unit on the final number.
If a physics problem involves mass in grams, distance in centimetres and force in Newtons, you must convert the units to a standard system (like SI units) before doing the arithmetic. AI models often miss this crucial step. Check the middle steps of the AI's response. Did it convert 50 g to 0.05 kg before plugging it into the formula? If it simply multiplied 50 by an acceleration and labelled the result "Newtons", the entire calculation is off by a factor of 1,000. Always trace the units manually from the first line to the last.
For a deeper look at verifying physics calculations and catching these subtle errors, see our post on when AI gets physics wrong: how to check an AI's solution.
Does the final number pass a sanity check?
A sanity check is your last and best defence against a hallucinated mathematical answer. This means pausing to look at the final result and asking: "Does this number make physical sense in the real world?"
If an AI calculates the speed of a car on a highway as 300,000 m/s, or a student's mass as 2,000 kg, it has made a mathematical error, no matter how confident the explanatory text sounds. Always estimate the rough order of magnitude in your head before looking at the AI's final answer. If you expect a fraction of a metre and the AI gives you three kilometres, you immediately know its logic is flawed.
If you are stuck on a specific calculation and want to be sure of the logic without second-guessing every line, the Fermi Problem Library has worked problems with a step-by-step solver that explains the reasoning rather than printing an answer. This allows you to verify the physics at each stage rather than blindly trusting the final result.
