On this page
You see the same three words on almost every study app: AI-powered, adaptive, personalised. They sound like a private tutor in your pocket. Often they mean something much smaller. Here is how to tell before anyone pays.
What does AI-powered actually mean in an edtech app?
The OECD describes an AI system as a machine-based system that infers, from the input it receives, how to generate outputs such as predictions, content, recommendations or decisions (OECD.AI, 2023).
That covers a lot: a rule that recommends the next chapter, a chatbot that writes explanations, a tutor that reads your working and finds the wrong step. The label says the software guesses something from your data. It does not say what, or whether it helps you learn.
What should adaptive and personalised mean?
Adaptive should mean the app changes what it does next because of how you answered: an easier step, a different explanation, or a question aimed at your gap. This kind of tutoring software has research behind it. A meta-analysis of 50 controlled evaluations of intelligent tutoring systems found a median gain of 0.66 standard deviations in test scores, roughly from the 50th to the 75th percentile (Kulik and Fletcher, Review of Educational Research, 2016). The same review warns that results depended heavily on whether gains were measured on locally developed or standardised tests, and that the few evaluations with flawed implementations found small effects.
Personalised is vaguer. Sometimes it means the adaptive behaviour above. Sometimes it means a sign-up quiz that labels you a visual or auditory learner. A major review found virtually no evidence that matching teaching to learning styles improves learning (Pashler and colleagues, Psychological Science in the Public Interest, 2008); we explain why in the learning styles myth.
Why can't you trust edtech marketing on its own?
Independent evidence is rare. UNESCO's 2023 Global Education Monitoring Report found little robust evidence on digital technology's added value in education (UNESCO GEM Report summary, 2023). In the United Kingdom, only 7% of education technology companies had conducted randomised controlled trials and 12% had used third-party certification, the same report says. It adds that a lot of the evidence comes from those trying to sell the product, and that edtech products change every 36 months on average, faster than they can be evaluated. A review of 23 primary-level maths apps it cites found they focused on drill and practice rather than advanced skills.
None of this means apps are useless. It means you need to test the product yourself.
Which questions reveal what a study app really does?
What happens when I get something wrong? A strong answer: it finds the step where you went wrong and responds to that step. A weak answer: it shows the right answer and moves on.
What exactly changes from one student to the next? Ask for a concrete example. If only the order of videos changes, adaptive means playlist.
Does it make me think, or think for me? Check whether it gives hints and asks you questions before revealing a full solution. Our guide on how to evaluate an AI tutor has a longer version of this test.
What is the evidence, and who produced it? Ask whether any study compared students using the app with students who did not, on a test the company did not write. Testimonials and toppers' photos are not evidence.
What does it do with my data? The UNESCO report cites an analysis finding that 89% of 163 education technology products recommended during the pandemic could survey children (UNESCO GEM Report summary, 2023). Read the privacy policy, and use our checklist on privacy in learning apps.
How do you test a study app in 15 minutes?
Use the free trial on a topic you half know, such as projectile motion.
Solve one problem correctly and note what the app does.
Solve a similar problem with a deliberate mistake, such as dropping a minus sign. Does it point to that step, or only mark the final answer wrong?
Type that you do not understand why. Does the explanation change, or does it repeat itself?
Parents: put question 4 to the salesperson word for word, and write down the reply.
If the app passes the first three checks, its adaptive claim means something. If it fails them, you are paying for a content library with a new label, which is fine only if the price matches.
This is the test we hold ourselves to. Fermi's AI tutor is built to notice where a student is stuck and decide whether to explain, ask or show, so you can run the same 15-minute check on it. For the wrong-step test, try a problem in the Fermi Problem Library, where a step-by-step solver explains the reasoning rather than printing an answer.
