On this page
Picture two versions of the same chemistry test. In the closed-book one, you write from memory, and a half-remembered fact sounds just as confident as a solid one. In the open-book one, you first find the right page, then write your answer from what it says. Retrieval-augmented generation, or RAG, is the idea of turning an AI chatbot from the first student into the second.
What is retrieval-augmented generation in simple words?
RAG is a way of building an AI system so that it looks up relevant passages from a collection of documents before it answers, and then writes the answer from those passages. The name comes from a 2020 paper by Patrick Lewis and colleagues, presented at NeurIPS 2020. It describes joining two kinds of memory:
What the model remembers: patterns soaked up from its training text. This is fixed once training ends and carries no page numbers.
What the model can look up: a separate library of documents (in the paper, an index of Wikipedia) that can be searched, and updated, at any time.
If you have wondered whether a chatbot is searching or remembering when it answers you, we unpack that in does AI search the web or remember?
How does RAG work, step by step?
You ask. For example: "When is our Class 11 chemistry practical, and what do we need to bring?"
Retrieve. A search step looks through the documents it has been given, say your school's circulars, and picks the few passages that best match the question. Many systems match by meaning rather than exact words, using embeddings, which place similar ideas close together.
Augment. Those passages are pasted into the model's input alongside your question, usually with an instruction such as "answer only from these sources".
Generate. The model writes the answer from the passages and, in a well-built system, shows which passage each part came from.
No model could have memorised your school's practical date during training. Without retrieval, it can only guess; with retrieval, it can read the circular.
Why does RAG cut made-up facts?
A language model with nothing to look up fills gaps with text that sounds right, which is how you get invented dates, formulas and even made-up sources. Giving it the relevant page changes the task from recalling to reading.
The research backs this up. The Lewis et al. paper reported that RAG models generate "more specific, diverse and factual language" than a comparable model working from memory alone. A 2021 study by Kurt Shuster and colleagues found that adding retrieval to chatbots substantially reduced knowledge hallucination, as verified by human evaluations.
Two bonuses: the answer can point to its source so you can check it, and the library can be updated without retraining the model.
Where does RAG still fail?
Open-book is not the same as correct. In the first preregistered evaluation of AI legal research tools, a Stanford-led team including Varun Magesh and Daniel Ho found that RAG-based tools from LexisNexis and Thomson Reuters each hallucinated between 17% and 33% of the time. That was fewer hallucinations than a general-purpose chatbot (GPT-4), but it showed that some providers' claims of "hallucination-free" answers were overstated.
The common failure points:
Wrong page. The search step pulls a passage that looks similar but answers a different question, and the model writes confidently from it.
Right page, misread. In the "Lost in the Middle" study, Nelson Liu and colleagues found models often used information best when it sat at the start or end of their input, and did significantly worse when it was in the middle of a long input.
Bad library. The answer can only be as good as the documents. An outdated PDF or an unofficial note gives a well-cited wrong answer.
Nothing found. If the library has no answer, a careless system falls back on memory instead of saying "I don't know".
How should you check an answer from a RAG tool?
Use this three-question check whenever an AI answer comes with sources:
Is it the right kind of source? NCERT, an official NTA or CBSE page, or your school's own circular beats a forum post or a screenshot.
Does the passage actually say this? Find the sentence. If you cannot, treat the claim as unverified.
Is it current? Exam dates, syllabi and rules change. Check the official page, not last year's PDF.
You can also ask for the open-book behaviour directly:
"Answer only from the sources you retrieved. For each point, quote the sentence you used. If the sources do not say, tell me that instead of guessing."
Finding the right passage is only half of studying; understanding it is the other half. Fermi's AI tutor teaches rather than just answers: it notices where you are stuck and decides whether to explain, ask or show. If a passage leans on an idea you have not met yet, the Fermi Concept Library has a lesson for each concept in physics, chemistry, biology, mathematics and computing.
Next time an AI hands you an answer with a source attached, open the book and check the page.
