On this page
Say you have a chatbot open in one tab and a computer algebra system (CAS), such as the free Python library SymPy, in another. Both will give you an answer to an integral. They get there in very different ways, and that changes how far you should trust each one.
What is the difference between a computer algebra system and a chatbot?
A computer algebra system works on symbols using fixed rules. SymPy's own tutorial says symbolic computation represents mathematical objects "exactly, not approximately", and that tools like it are "often called computer algebra systems" (SymPy tutorial). Its example: ordinary Python gives √8 as 2.82842712475, while SymPy returns 2√2.
A chatbot is a large language model. It writes the most likely next piece of text based on everything before it, and that text is often right. But nothing inside the model checks that line 4 really follows from line 3. Our explainer on how large language models work covers the mechanics.
Why do chatbots make maths mistakes a CAS would not?
The GSM-Symbolic study tested language models on school-level word problems and then changed only the numbers. Performance dropped (Mirzadeh et al., 2024). When the researchers added a single clause that seemed relevant but did nothing to change the answer, performance dropped by up to 65% across the state-of-the-art models they tested. The authors suggest that current models cannot perform "genuine logical reasoning" and instead "replicate reasoning steps from their training data". We go through the common failure patterns in why AI chatbots get maths wrong.
One fix researchers found is to give the calculation to software. In the PAL method, the language model writes a short program and a Python interpreter runs it. On the GSM8K word-problem benchmark, PAL running on OpenAI's Codex beat Google's PaLM-540B model reasoning step by step in plain text by 15 percentage points (Gao et al., 2022).
A CAS has blind spots too. It answers exactly what you typed, so a missing bracket gets you an exact answer to the wrong question. It also skips things your teacher expects. SymPy's documentation notes that it "does not include the constant of integration" (SymPy calculus tutorial). And it rarely tells you why a method works.
Which one should you trust for algebra and calculus?
Match the tool to the job:
Expanding, factorising and simplifying: use the CAS.
Derivatives, integrals, limits and solving equations: use the CAS for the final result.
"Why does this substitution work?" or "Which method should I try?": ask a chatbot or your teacher, then test what they tell you.
Word problems: let a chatbot help you set up the equation, solve it with the CAS, then check the answer makes sense yourself.
Checking your own handwritten solution: put each line into the CAS separately, so you find the exact step that broke.
In the exam: neither. Practise so your hand can do it alone.
How do you use both to check your work?
Here is a JEE-style integral: ∫ x / ((x − 1)(x + 2)) dx. If partial fractions are rusty, see our guide to integration by partial fractions.
Solve it yourself first. Write x = A(x + 2) + B(x − 1). Put x = 1 to get A = 1/3, and x = −2 to get B = 2/3. Your answer: (1/3) ln|x − 1| + (2/3) ln|x + 2| + C.
Suppose a chatbot gives (1/3) ln|x − 1| − (2/3) ln|x + 2| + C. It looks right, and only one sign is different.
Differentiate, don't re-integrate. Ask the CAS for the derivative of each answer minus the original integrand. In SymPy that is simplify(diff(answer, x) - x/((x-1)*(x+2))), typing ln as log and leaving out the absolute-value bars. The correct answer gives 0.
No CAS to hand? Spot-check at x = 2. The integrand is 2 / (1 × 4) = 0.5. The derivative of your answer is 1/3 + (2/3)(1/4) = 0.5. The chatbot's gives 1/3 − 1/6 = 1/6, so it is wrong.
Now ask the chatbot where its minus sign came from. Explaining a mistake is a good use for it.
What should you still do yourself?
If you can't reproduce an answer, you haven't learned it, however exact it is. Write your method first and use the tools to check it. Parents, the useful question at the table is not "Did you use AI?" but "How did you check it?"
If your answers keep failing because the method is wrong, not because of a slip, look at a full worked solution. The Fermi Problem Library has worked problems with a step-by-step solver that explains the reasoning rather than printing an answer. And if you want one place that explains, asks and shows as you work, Fermi's AI tutor holds a line of reasoning across the whole session and notices where you are stuck.
Tonight, check one integral you have already solved by differentiating it.
