On this page
You do not need to understand engines to drive, but knowing that brakes wear out makes you a safer driver. Large language models (LLMs), the technology behind ChatGPT, Gemini, Claude and similar tools, are the same. A simple, accurate picture of how they work explains why they are so good at explaining, why they sometimes make things up, and how you should use them for study.
What is a large language model, in one sentence?
It is a program trained to predict what text comes next, one small piece at a time, and it becomes useful because it was trained on an enormous amount of text.
The GPT-3 paper described its model as "an autoregressive language model with 175 billion parameters" (Brown and colleagues, 2020). Autoregressive means it writes one piece, adds it to the text so far, then predicts the next piece from all of that. Parameters are the adjustable numbers inside the model that get tuned during training.
Try it yourself. Finish this sentence: "The acceleration due to gravity near Earth's surface is about..." You probably thought "9.8 metres per second squared". You did not look it up; you have seen that pattern many times. An LLM does something similar, but with patterns learned from far more text than any person reads, and without a separate fact-checking step built in.
What are tokens and attention?
Tokens. Models do not read text letter by letter. Text is split into chunks called tokens, which may be a word, part of a word or a few digits. This matters for maths: one study notes that GPT-3.5 and GPT-4 use separate tokens for one-, two- and three-digit numbers, and found that how numbers are split affects arithmetic accuracy (Singh and Strouse, 2024). A number like 12345 is not seen as five digits in a row, the way you would line it up for addition.
Attention. LLMs are built on the transformer, an architecture introduced in the 2017 paper "Attention Is All You Need" (Vaswani and colleagues). Attention lets the model weigh which earlier tokens matter most when predicting the next one. In "The ball was thrown up; at the top of its path its velocity is...", attention can link "top of its path" to "velocity", the kind of connection that helps the model produce "zero" for the vertical component.
How does a predictor become a helpful assistant?
Raw next-word prediction produces fluent text but not necessarily helpful answers. So models get a second stage of training. In the InstructGPT work, people wrote example answers and ranked model outputs, and the model was tuned on those preferences (Ouyang and colleagues, 2022). The result was striking: in human evaluations, outputs from a 1.3-billion-parameter tuned model were preferred to those from the 175-billion-parameter GPT-3. This kind of tuning is a big part of why chatbots answer in a friendly, structured, explanatory way.
Why do LLMs make things up?
Because predicting plausible text is not the same as knowing what is true. A 2025 paper argues that models produce confident falsehoods partly because "the training and evaluation procedures reward guessing over acknowledging uncertainty" (Kalai and colleagues, 2025). They compare it to a student guessing on a hard exam question rather than admitting uncertainty. Fluent, confident wording tells you nothing about whether an answer is correct.
What does this mean for how you study?
Each part of the picture gives you a practical rule.
Because it predicts plausible text: use it for explanations, analogies and questions, and check facts, constants and formulas against your textbook or NCERT. For citations, see when AI makes up sources.
Because numbers are split into tokens: do not trust long arithmetic done in its head. Ask it to show each step, or check with a calculator. We explain this in why AI chatbots get maths wrong.
Because attention works on what you give it: context helps. Paste the full question, your working and where you got stuck, rather than "solve this".
Because it was tuned to be helpful: it will happily hand over full answers. If you want to learn, say so: "give me a hint, not the solution".
Because it may guess rather than say "I don't know": ask "how confident are you, and what would change your answer?" and treat any disagreement with your textbook as a reason to check.
This is why the design of a study tool matters as much as the model inside it. We built Fermi's AI tutor to teach rather than answer: it holds a line of reasoning across a session and checks understanding as it goes.
A two-minute experiment to try
Ask any chatbot the same physics question twice in fresh chats, with slightly different wording. If the answers differ, you have seen prediction at work. Then ask it to explain the difference. Spotting and resolving that disagreement is good physics practice in itself, and a habit worth keeping, as in our guide to evaluating an AI tutor.
