On this page
Students increasingly turn to chatbots for help when they get stuck on their physics and chemistry homework. Thermodynamics is notoriously difficult, filled with subtle distinctions between state functions, path functions, and reversible processes. However, when it comes to this subject, language models often stumble. A recent study evaluating the UTQA (Undergraduate Thermodynamics Question Answering) benchmark found that the best AI models achieve only 82% accuracy, falling significantly short of the reliability needed for unsupervised tutoring.
Because chatbots generate text statistically rather than computing physical laws, they frequently mix up conventions or lose track of units mid-problem. Here is how to verify their output.
How to check an AI thermodynamics numerical answer?
When reviewing an AI's step-by-step solution, you should never accept the final number at face value. Instead, follow this four-step framework to spot the most common hallucinations.
1. Check the sign convention for work (w) AI models frequently confuse the chemistry and physics rules for whether work is positive or negative. The IUPAC sign convention, standard in chemistry, treats the system like a bank account. Under this rule, work done *on* the system is positive, and work done *by* the system (such as during a gas expansion) is negative (w < 0). In traditional physics and engineering, the convention is reversed, and work done *by* the system is considered positive. Verify that the AI is using the correct First Law equation for your subject (ΔU = q + w for chemistry vs ΔU = q - w for physics) and sticking to it without silently swapping signs.
2. Verify the units of the gas constant (R) Chatbots often grab the wrong numerical value for R depending on the text immediately preceding it in the prompt, ruining the calculation. Look closely at the number the AI chose:
8.314 J K⁻¹ mol⁻¹: Use this value for all energy equations and when the pressure is in Pascals and the volume is in cubic metres.
0.0821 L atm K⁻¹ mol⁻¹: Use this value only when working directly with litres and atmospheres.
If an AI model uses 8.314 but leaves the gas volume in litres, its final numerical answer will be off by a massive factor of 100. Always check that the R value aligns with the units in the current step.
3. Look for the ΔT trap in Kelvin conversions In thermodynamics, an absolute temperature (T) must always be converted to Kelvin by adding 273.15 to the Celsius value. However, a temperature *difference* (ΔT) is exactly the same in both the Celsius and Kelvin scales. A 15 °C increase is precisely a 15 K increase. AI models routinely make the mistake of adding 273.15 to a ΔT, severely corrupting equations like q = nCpΔT used for finding heat in isobaric and isochoric processes.
4. Confirm the process assumptions AI answers frequently hallucinate implicit assumptions that contradict the question. For example, a chatbot might incorrectly assume a process is isothermal (constant temperature) simply because a starting temperature was given, or it might treat a real gas as an ideal gas. Always verify that the model's chosen formulas align with the explicitly stated conditions in the problem.
Why do AI models mix up physics and chemistry rules?
Large language models do not possess an internal calculator, nor do they hold a working model of the physical world. Instead, they predict which words or mathematical symbols are most likely to follow the previous ones based on patterns in their training data.
Because concepts like the net work in a cycle are taught differently in physics and chemistry textbooks, both conventions are heavily represented in the AI's training text. When answering your question, the model might correctly interpret a chemistry prompt at the start, but then "remember" a derivation from a physics textbook halfway through, causing it to swap its signs and reach a flawed conclusion.
What is a better way to study thermodynamics?
If you rely on a standard chatbot to simply give you a final number, you risk internalising its calculation errors and taking false assumptions into your exams. Instead, you should use study tools that are designed to actively build your understanding of the underlying concepts.
Fermi's AI tutor is built to teach rather than just print an answer. It holds a line of reasoning across a session, noticing where you are stuck and deciding whether to explain, ask, or show. If you are confused about whether the work in irreversible expansion should be positive or negative, Fermi will ask you to think about the system's energy balance to guide you to the right convention. When you need to see exactly how a specific calculation is structured, the Fermi Problem Library has worked problems with a step-by-step solver that explains the reasoning rather than printing an answer.
