On this page
Ask a chatbot for sources and it will often give you a neat list of authors, titles, journals and years. Some of those papers may not exist. The list looks just as convincing either way, which is why checking matters for any project, essay or science fair report.
How often are AI references fake?
Walters and Wilder asked ChatGPT-3.5 and GPT-4 to write short literature reviews on 42 topics and then checked all 636 citations. 55% of the GPT-3.5 citations and 18% of the GPT-4 citations were fabricated. Among the real ones, 43% of GPT-3.5's and 24% of GPT-4's had substantive errors. GPT-4 did much better, but the authors concluded that "problems remain."
The stakes can be high. In the US case Mata v. Avianca, lawyers filed a brief citing court decisions that ChatGPT had invented. On 22 June 2023, Judge P. Kevin Castel ordered the two lawyers and their firm to pay a $5,000 penalty, writing that they had "abandoned their responsibilities when they submitted non-existent judicial opinions with fake quotes and citations created by the artificial intelligence tool ChatGPT".
Why does this happen?
A language model produces text that fits the patterns it has learned. A reference has a very regular pattern: names, year, title, journal, volume, pages. The model can produce something that fits the pattern without retrieving a real record. Our explainer on how large language models work covers why. Tools that search the web and link to real pages reduce the problem, but you still need to open the link.
A five-step check for any reference
Search the exact title. Put it in quotes in Google Scholar or Crossref's search. No match anywhere is a strong warning sign.
Resolve the DOI. If there is a DOI, open https://doi.org/ followed by the DOI. For a quick data check, Crossref's free API at api.crossref.org/works/ followed by the DOI returns the title, authors, journal and date.
Match every detail. Title, first author, journal and year should all agree. A real DOI attached to a different paper is a common error.
Read the abstract. Does it support the exact claim, including the numbers? If the AI says "a 40% improvement" and the abstract doesn't, don't use the number.
Go to the primary source for facts that change. Exam patterns, syllabi and dates belong to official pages, such as NTA for JEE Main and NEET-UG or CBSE for board exams. Not a chatbot, and not a blog.
A worked check
Here is how we checked the Walters and Wilder study for this article. The DOI 10.1038/s41598-023-41032-5 resolved on Crossref to the title "Fabrication and errors in the bibliographic citations generated by ChatGPT", in Scientific Reports, published 7 September 2023, with the authors Walters and Wilder. The abstract gave the 42 topics, 636 citations and the 55% and 18% figures used above. Every detail matched, so we cited it. A check like this takes a few minutes.
Red flags in an AI reference list
A DOI that doesn't resolve, or resolves to a different paper.
A real journal but a title you can't find in it.
Very round numbers, or quotes with no page or link.
A famous author paired with a paper outside their field.
Every source conveniently agreeing with the point being made.
A better way to use AI for research
Ask the AI for search terms, key ideas and the names of review articles to look for, then find the sources yourself. Once you have a real paper, paste its abstract into the chat and ask questions about it. That way the AI works on text you have verified. Our guides on AI and homework honesty and checking you understood an AI explanation go further.
Where Fermi fits
When what you need is to understand an idea rather than to cite it, the Fermi Concept Library has a lesson for each concept in physics, chemistry, biology, mathematics and computing. For a project, you still need sources you have opened and checked yourself.
