On this page
Most AI tutoring studies come from countries where every student has a laptop. A study from Ghana tested something closer to what many Indian students have: a basic phone and a chat app. This post explains what the Rori study did, what it found and why it should be read as early evidence.
What question did the Rori study ask?
The paper is Effective and Scalable Math Support: Experimental Evidence on the Impact of an AI-Math Tutor in Ghana by Owen Henkel, Hannah Horne-Robinson, Nessie Kozhakhmetova and Amanda Lee. It is a preprint on arXiv, which means it has not been through journal peer review.
It asked whether extra maths practice with Rori, an AI maths tutor available on WhatsApp, would improve maths scores compared with normal schooling.
What is Rori?
Rori was built by Rising Academies, a school network. According to the paper, students work through a curriculum of over 500 micro-lessons based on the Global Proficiency Framework, an international set of maths learning standards. Each micro-lesson has a short explanation and scaffolded practice questions. A student who struggles gets a hint first and then a worked solution. Language models and other natural language processing methods let students type answers and questions in ordinary language. The authors say it was inspired by Teaching at the Right Level and intelligent tutoring systems.
Who was studied, and how?
Students in grades 3 to 9 from 11 Rising Academies schools in Ghana took part. Schools were randomly assigned: five to the treatment group and six to the control group. Assignment was at the school level because assigning individual students was not feasible.
Treatment students used Rori for two 30-minute sessions a week during study hall, on top of regular maths classes. Teachers were present only to help with technical problems. Control students had their normal study hall without Rori.
Both groups took the same 35-question maths test in early February and late August 2023, covering numeracy and algebra skills from grades 3 to 5. Of 637 students who took the baseline, 477 also took the endline: 241 in the control group and 236 in the treatment group. The 160 who dropped out mostly missed school.
What did it find?
Growth scores were calculated as endline minus baseline correct answers.
Control group: average growth of 2.12 questions.
Rori group: average growth of 5.13 questions.
Difference: 3.01 questions, statistically significant (p < 0.001).
The authors report an effect size (Cohen's d) of 0.36. They describe this as roughly equal to or more than a year of schooling, comparing the Rori group's extra growth with the control group's growth. They estimate the marginal cost at about $5 per student per year.
What does the study not show?
The authors list several limits, and they matter:
Only 11 schools were randomised. With so few units, chance differences between schools can affect results more than in a student-level trial. The authors note possible unobserved differences and a possible Hawthorne effect, where people behave differently because they are being observed.
The test had a ceiling. The same grade 3 to 5 test was used for all students, so some older students scored full marks at baseline and could not show gains. The authors note it is also possible Rori works mainly for easier topics.
One year, one school network. These are year-one results from schools already used to testing and data.
It is a preprint. Peer review may change the analysis.
What does it mean for Indian students?
Two things transfer to Indian conditions. First, a tutor does not need a laptop or fast internet to help; short, regular practice on a phone can add up. Second, the design copied what works in human teaching: start at the right level, give a hint before the answer, and keep sessions short and regular.
It does not mean any chatbot will have the same effect. Rori was built around a structured curriculum and graded hints. Studies of general chatbots that hand out answers have found different results; our post on AI tutors in real classrooms covers several of them, and hints or answers explains why the type of help matters.
How to read a study like this
Check three things: whether the study is peer reviewed, what was randomised (students or schools), and whether the test was hard enough to show improvement for everyone. A preprint with school-level randomisation and a ceiling on the test can still be useful, but it is a first step, not the final word.
If you want AI help that teaches rather than hands over answers, Fermi's AI tutor notices where you are stuck and decides whether to explain, ask or show, and checks your understanding as it goes.
Try two 30-minute practice sessions this week on one maths topic, with hints before answers, and compare your score before and after.
