On this page
You ask a chatbot to summarise an NCERT English chapter the night before a test. The summary reads smoothly, so you revise from it. Then in class you find that a scene you learned never happens in the book. This is a known weakness of AI summaries, and a page check catches it.
Why does an AI summary of an English chapter have wrong events?
Researchers call made-up content in a summary a hallucination. A human evaluation by Joshua Maynez and colleagues found substantial amounts of hallucinated content in all model-generated summaries they examined.
Stories are a hard case. The FABLES study had people check 3,158 claims from AI-written summaries of 26 recently published works of fiction. Most unfaithful claims were about events and character states, and catching them usually needed reasoning across the story, not one glance. The same study found that summaries over-emphasised events near the end of the book.
Two more findings explain why the errors sound so sure. A 2025 paper by Adam Tauman Kalai and co-authors argues that language models hallucinate because training and evaluation reward guessing over admitting uncertainty. And the Lost in the Middle study found that models used information at the start or end of a long input better than information in the middle. It tested question answering and key-value retrieval, not stories, so treat details from the middle of a chapter with extra care.
There is a simpler problem too. If you type only the chapter title, the chatbot has no page in front of it. It writes from whatever it absorbed about the story, and it can fill gaps with what usually happens in stories like it.
How do you check an AI summary against the NCERT textbook?
Use a claim-to-page check:
Download the chapter PDF free from the NCERT textbooks page, or open your printed book.
Break the summary into single claims, one per line: who did what, where, when.
Next to each claim, write the page number where the book says it. Search the PDF for names to go faster.
Mark each line: a page number means keep; a tilde (~) means partly right, so rewrite it in the book's words; a question mark means no page, so cross it out.
Revise only from lines that carry a page number.
What does an invented detail look like in a real chapter?
Take The Last Lesson by Alphonse Daudet, the first prose chapter in the Class 12 Flamingo textbook. Below is a sample summary we wrote to show the typical errors (it is not the output of any one chatbot), checked against the printed page numbers:
Franz is late for school and dreads being questioned on participles. Page 2: keep.
A crowd stands in front of the bulletin-board at the town hall. Page 2: keep.
Franz's mother wakes him and warns him not to be late. No page: there is no such scene. The only mention of his parents is M. Hamel saying they were “not anxious enough to have you learn” on page 5.
The order from Berlin says only German will be taught in the schools of Alsace and Lorraine. Page 4: keep.
Village elders, including old Hauser with an old primer, sit on the back benches. Page 4: keep.
Prussian soldiers march into the classroom and arrest M. Hamel. No page: on page 8 the trumpets of the Prussians, returning from drill, sound under the windows. Nobody enters, and nobody is arrested.
M. Hamel writes “Vive La France!” on the blackboard. Page 8: keep.
Notice that each invented line grows from something real: parents who did not push Franz, soldiers drilling nearby. That is why they feel right, and why errors like these take reasoning across the story to catch.
What should you ask the AI instead?
Give it the text, and ask for pages. Paste one page at a time and use this prompt:
“Summarise only the text I pasted. Do not add anything from memory. After each sentence, put the page number in brackets. If something is unclear, write: not in the text.”
Then still run the check, because page numbers can be made up too. A stronger habit is to write your own five-line summary first, then ask the AI to point out any line the text does not support.
A guess that sounds certain is still a guess; this post on whether an AI's confidence tells you it is right explains why. The same habit works when you interview a scientist with AI and fact-check every answer. Before your boards, also read about writing AI-worded answers in board exams.
For science chapters, we built Fermi's AI tutor to teach rather than hand over a finished answer: it holds a line of reasoning across a session and checks understanding as it goes. Whatever tool you use, the page has the final word.
