On this page
Generating your own practice material can be a great way to study, but it comes with a hidden risk: artificial intelligence often invents questions that are mathematically impossible to solve.
If you have ever spent an hour trying to balance a physics equation only to realize a crucial variable is missing, the problem might not be your understanding. It might be the prompt.
Why does an AI made practice question have no answer?
An AI does not build a question the way a human teacher does. A human educator starts with an answer, works backwards to build a logical path, and then tests it. An AI predicts text based on patterns. When you ask it to generate a difficult question, it strings together impressive-sounding conditions. Often, it drops an essential variable in the process, accidentally severing the logical path to the answer.
This is a well-documented failure mode for large language models. A 2025 research dataset named TreeCut was developed specifically to test how models handle problems with missing data. The study (arXiv:2502.13442) showed that when necessary facts are deliberately removed from a math problem, AI models still guess a confident, numerical answer in up to 64% of worst-case scenarios, rather than noticing the gap. The AI simply hallucinates the missing number behind the scenes and presents a completely fabricated solution.
How do you know if a math question has no answer?
Before you spend time trying to solve a generated question, run a quick structural check. To see if a math or physics question can be solved, you must compare the unknowns to the constraints.
Count the unknowns: These are the variables you need to find, plus any intermediate variables you do not know.
Count the constraints: These are the independent equations, formulas, or data points given in the text.
Compare them: If the number of constraints is less than the number of unknowns, the problem is missing data and cannot be solved.
For example, imagine an AI gives you this word problem: "3 eggs cost $4 less than 2 burgers. How much does an egg cost?" You have two unknowns (the price of an egg and the price of a burger) but only one constraint (the equation connecting them). Because there is one more unknown than there are constraints, it is mathematically indeterminate. No amount of algebra will yield a real number without additional information.
What if the given data is contradictory?
Sometimes an AI gives you too much information, making the problem overconstrained and contradictory. For instance, an AI might ask you to find the area of a right-angled triangle with sides of 3 cm, 4 cm, and 6 cm. However, 3² + 4² equals 25, not 36. Any time spent trying to apply trigonometric formulas to this shape is wasted because the geometry itself is impossible.
Before spending an hour on a geometry or algebra problem, run a quick sense-check on the initial conditions. For instance, you might want to review the discriminant test for when ax² + bx + c is positive for all x to ensure the AI has not given you a quadratic equation with impossible roots. If you are calculating rates of change, ensure the variables actually depend on each other before differentiating one expression with respect to another.
How can you safely prompt AI for practice problems?
If you still want to use AI to generate questions, you can adjust your prompts to force the model to verify its own work. Instead of simply asking for a difficult problem, ask the AI to generate the full step-by-step solution first, and only present the question to you if the final answer is a whole number.
You can also ask the AI to list all the variables required to solve the problem before it writes the question text. This forces the model to define its constraints clearly. However, because language models fundamentally struggle with mathematical logic, these tricks will only reduce the error rate, not eliminate it entirely.
How can you get better practice questions?
Instead of endlessly arguing with a chatbot to fix its own word problems, use platforms that rely on vetted, human-verified questions. For example, the Fermi Problem Library has worked problems with a step-by-step solver that explains the reasoning rather than printing an answer, so you never waste time on a hallucinated scenario.
When you need guidance on how to approach these problems, look for an AI that behaves like a true educator. Fermi's AI tutor teaches. It holds a line of reasoning across a whole session, notices where you are stuck, and decides whether to explain, ask, or show. It works out what you already know and builds a plan for the session, rather than just predicting the next word.
