On this page
For years, claims about AI in education ran ahead of the evidence. In the past few years, a handful of randomised controlled trials have tested AI tutors with real students in real courses. The results are encouraging, but each trial tested a particular design in a particular setting. Here is what four of them found, what they share, and what nobody knows yet.
Harvard physics: an AI tutor vs an active-learning class
Gregory Kestin, Kelly Miller and colleagues ran a trial with 194 students in Physical Sciences 2, a physics course for life sciences majors at Harvard, in fall 2023 (Harvard Gazette). Over two consecutive weeks, one group had an instructor-led active-learning lesson while the other used a custom AI tutor at home, and then the conditions were swapped, so every student experienced both. The tutor was built on the GPT API with research-based instructions written for each lesson, rather than default chatbot behaviour.
The published paper reports that students learned significantly more in less time with the AI tutor, and felt more engaged and more motivated (Kestin et al. 2025, Scientific Reports). The Gazette, reporting a preliminary analysis, described learning gains about double those in class. Note the comparison: this was not AI against a dull lecture, but against a class already using active learning.
Nigeria: an after-school programme with teachers in the room
In June and July 2024, a World Bank team ran a six-week after-school programme in Edo State, focused on English, with students randomly assigned to take part or not (World Bank education blog). Public senior secondary students attended twice a week, and teachers were trained to guide them, keep them on task and spot AI errors (World Bank Development Talk).
Students improved by about 0.3 standard deviations, which the team estimates as roughly 1.5 to 2 years of typical learning. Each extra day of attendance brought more learning. Girls, who started behind, seemed to gain more, and participants also did better on their end-of-year exams. The authors are careful: they "cannot isolate the effect of the chatbot alone", because the result reflects the full package of prompts, curriculum, teachers, peers and AI.
High school maths: when AI becomes a crutch
Not every result is positive. In a field experiment with nearly a thousand high school maths students, a plain ChatGPT-style assistant raised practice grades by 48%, but once it was taken away, those students scored 17% lower than students who never had access (Bastani et al. 2025, PNAS). A tutor version with prompts designed to safeguard learning largely avoided that harm. The design of the tool, not its mere presence, decided the outcome.
Human tutors helped by AI
A fourth trial put AI behind a human tutor instead of in front of the student. In a study of 900 tutors and 1,800 K-12 students, tutors with an AI assistant called Tutor CoPilot were more likely to ask guiding questions and less likely to give away answers, and their students were 4 percentage points more likely to master topics, with the largest gains for students of lower-rated tutors (Wang et al. 2024). It is posted as a preprint on arXiv, so treat it as early evidence.
What we still do not know
Lasting effects. These trials were short, lasting weeks or months rather than years. Whether gains hold across two years of exam preparation is untested.
Other subjects and levels. One trial was university physics, one school English, and two were school maths or tutoring. Results may differ for JEE-level problem solving.
Novelty. Students may engage more with something new. Longer studies will show whether that fades.
Scale. A carefully designed tutor in one course is not the same as any AI app used without guidance.
For context, an earlier review of computer tutoring found intelligent tutoring systems nearly as effective as human tutors, with effect sizes of 0.76 and 0.79 (VanLehn 2011). The new studies suggest generative AI can extend that line of work when it is designed with the same care.
What this means for students and parents
Judge a tool by how it behaves, not by the label "AI". Does it ask you to try a step before showing one? Does it check your understanding? Does it fit into a plan? Our eight tests for evaluating an AI tutor turn these questions into a 20-minute check, and AI tutors vs answer chatbots explains the difference in depth.
We designed Fermi's AI tutor to teach rather than answer: it holds a line of reasoning across a session, notices where you are stuck and checks understanding as it goes. The studies above tested other tools, not Fermi, and we will keep reporting the evidence as it arrives.
