On this page
In spring 2024, a team led by Rose Wang and Dorottya Demszky ran what their paper calls the first randomised controlled trial of a human-AI system in live tutoring (Wang et al., arXiv 2410.03017). The AI, Tutor CoPilot, never spoke to students; on request, it suggested to the human tutor what to say next.
For anyone who runs or pays for tutoring, the result is encouraging but modest: the AI helped a little overall, and most of the help went to the tutors who needed it most.
What did the Tutor CoPilot study test?
The trial ran for about seven weeks, from 27 March to 17 May 2024, in chat-based online maths tutoring for grades 3 to 6, delivered by FEV Tutor in a southern US school district; 67% of the student sample was economically disadvantaged (study, v3). Tutors, not students, were randomly given access to the tool or not.
The analysis covered more than 700 tutors and about 1,000 students, according to the working paper version (EdWorkingPaper 24-1056, DOI 10.26300/81nh-8262).
Mid-session, a tutor clicks a button and picks a strategy, such as giving a hint or showing a worked example. Tutor CoPilot drafts a message from the de-identified chat, the lesson topic and the chosen strategy, and the tutor can edit it, regenerate it or pick a different strategy (study, v3).
Does AI help human tutors teach better?
In this study, yes, by a measurable amount. In the latest version of the paper, 61.7% of students in the control group passed their exit tickets, and students whose tutors had Tutor CoPilot were 4 percentage points more likely to pass (study, v3).
The teaching itself changed. In an analysis of the chat messages, tutors with access asked more guiding questions and prompted students to explain their thinking more often, and were less likely to give away the answer (study, v1) or offer generic praise (working paper).
The authors put its cost at about $20 per tutor per year, against teacher professional development programmes they cite at $3,300 or more per teacher a year, and tutors used it in about 29% of sessions without any reminders (study, v1).
The limits matter too. The researchers found no statistically significant improvement in end-of-year maths test scores, and tutors said some suggestions were not pitched at the students' grade level (study, v3).
Which tutors gained the most from AI suggestions?
The gains were concentrated at the bottom. Students of lower-rated tutors went from 56% mastery without the tool to 65% with it, a 9-point rise, and the effect shrank as tutor ratings and experience rose (study, v3).
Experience showed the same pattern: students of less experienced tutors rose from 61% to 68% mastery (same source). With the tool, these students did as well as, or better than, students of stronger tutors without it.
This is one seven-week study of US school maths, so treat it as a signal, not a rule.
How could a tutoring centre try an AI assistant?
If newer teachers handle some of your centre's doubt sessions, the study points to where an AI assistant may help most. A cautious pilot:
Start with your newest or lower-rated tutors, since that is where the trial found the biggest gains.
Set up the tool around teaching moves (hint, guiding question, worked example), not around producing final answers.
Keep the tutor in charge: every suggestion is checked against the class level and syllabus before sending.
Strip student names and contact details before anything reaches the AI.
Measure with a short exit check each session, comparing tutors with and without the tool.
Watch real usage, not licences bought: trial tutors used it in about 29% of sessions.
Here is the kind of change the study measured. A Class 11 student writes: "I solved x squared = 9 and got x = 3, but the answer key says x = 3 or x = -3."
Giving the answer: "You forgot the negative root. It is plus or minus 3."
Guiding question: "Try putting x = -3 into x squared. What do you get? So how many numbers square to 9?"
The second reply lets the student find the mistake. We explain why that matters in hints or answers: what tutoring research says about the right help.
What can students take from this?
The lesson is not that tutors need AI. It is that the right next move is often a question, not an answer. A related study compared ChatGPT hints with human tutor hints in maths.
If you also study with AI on your own, look for that same behaviour. Fermi's AI tutor decides whether to explain, ask or show, and checks understanding as it goes, so you do the thinking rather than copying a finished solution.
Next, read our guide to choosing between an AI tutor and a private tutor, and combining them.
