On this page
If you have read anything about "what works" in education, you have probably seen a ranked list of teaching strategies with numbers beside them. Many of those lists trace back to John Hattie's *Visible Learning*. The numbers are effect sizes, and they look precise. This post explains what Hattie did, works through what an effect size means, and sets out why some statisticians think the rankings mislead, so you can read such claims with care.
What is Visible Learning?
Hattie's book, published by Routledge, describes itself as the result of 15 years of research that "synthesises over 800 meta-analyses on the influences on achievement in school-aged students", covering many millions of students. It ranks influences from the student, home, school, curricula, teacher and teaching strategies on a common scale.
Its main message, in the book's own summary, is that what works for students resembles what works for teachers: challenging learning intentions, clarity about what success means, and attention to strategies for building understanding.
What is an effect size?
An effect size puts different studies on one scale by measuring a difference in units of spread. The common version is Cohen's d:
d = (mean of group A − mean of group B) ÷ standard deviation
Worked example. A class taught with a new method averages 68 marks; a similar class taught the usual way averages 62. The standard deviation of marks is 15.
d = (68 − 62) ÷ 15 = 6 ÷ 15 = 0.40
What does 0.40 mean in everyday terms? One way is the "common language" effect size: the chance that a randomly chosen student from group A scores higher than one from group B. For d = 0.40 it is about 61%, against 50% if there were no difference. A real effect, but the groups overlap heavily.
What is the 0.40 hinge point?
Hattie used 0.40 as a dividing line. As statistician Pierre-Jérôme Bergeron describes it in his critique of Visible Learning, effects below zero count as harmful, effects between 0 and 0.4 run from "developmental" to typical "teacher" effects, and effects above 0.4 form the desired zone. Bergeron argues there is no justification for this classification.
What do the critics say?
Bergeron's critique, published in the McGill Journal of Education, makes three main points. First, a calculation error: the common-language effect sizes in the book, as Norwegian researchers noticed in 2012, sometimes came out as negative probabilities or probabilities above 100%. Second, averages that mix incomparable things, such as studies of different durations, levels and populations. Third, converting correlations into Cohen's d without checking that the conversion is valid.
Adrian Simpson's analysis in the Journal of Education Policy goes further. He argues that a standardised effect size is open to research-design choices, such as the test used and the comparison group, which break the assumptions needed to compare and combine studies. League tables of interventions, he writes, "may, instead, be hierarchies of openness to research design manipulations".
Matthew Kraft, writing in Educational Researcher, adds a different warning: effects that are small by Cohen's old benchmarks are large compared with most real school interventions, and that cost and scalability belong in the judgement too.
Why does study design change the number?
Look again at the formula. The difference in means sits on top; the spread sits underneath. If a study tests a narrow, selected group, the spread is smaller and d grows. In the callout's example, the same six-mark gain is 0.40 with a spread of 15 but 0.60 with a spread of 10. The choice of test matters in the same way. Simpson's point is that decisions like these, which sample and which test, can move d without any change in the teaching.
Is Visible Learning still useful?
Yes, as a list of questions rather than answers. Its emphasis on feedback and clear goals fits other strong evidence, such as Black and Wiliam's work on formative assessment. The mistake is to read a ranking as a precise measure of what will work in your classroom or for your child.
How should students and parents read an effect size claim?
Ask what was compared with what. Better than nothing, or better than ordinary teaching?
Ask who was tested and on what. A narrow group and a matched test inflate d.
Ask how many studies agree. One striking study is weak; many consistent ones are stronger.
Look for overlap. Even d = 0.40 leaves many students in the "worse" group ahead of many in the "better" one.
Watch for neat stories. The myth of learning styles spread on intuition, not data.
Feedback during practice is the part of Hattie's message easiest to act on at home. Fermi's AI tutor notices where a student is stuck and checks understanding as it goes, so feedback arrives while the problem is still on the page.
Next time you see a ranked list of strategies, pick one item and find out what it was compared against before you believe the number.
