Skip to content

Great Minds

Black and Wiliam's 'Inside the Black Box': feedback that helps

Paul Black and Dylan Wiliam argued that everyday assessment used as feedback raises achievement. What they found, what later critics said, and how to use it.

By Fermi Editorial

4 min read

Teacher helping a child with written work at a table
Ronald Felton / Unsplash

In 1998, two researchers at King's College London, Paul Black and Dylan Wiliam, published a short booklet with a memorable title: *Inside the Black Box*. Their argument was that government policy kept changing what went into classrooms (tests, targets, inspections) while treating the classroom itself as a black box. Standards, they said, could only rise if teachers got direct help with what happens inside it, above all with how teachers and pupils use evidence of learning.

What is formative assessment?

The booklet summarised a longer review, "Assessment and Classroom Learning", published in *Assessment in Education*. In the booklet, Black and Wiliam say their search turned up about 580 articles or chapters, and the review drew on 250 of them.

Their definition is simple. Assessment includes everything teachers and students do that gives information to be used as feedback. It becomes *formative* when that evidence is actually used to adapt teaching to meet students' needs. A test that only produces a mark is not formative. The same test, used to decide what to reteach tomorrow, is.

What did Black and Wiliam find?

Four findings from the booklet stand out:

  • Size of effect. The formative assessment experiments they reviewed produced "typical effect sizes of between 0.4 and 0.7". They explained that an effect size of 0.4 would put the average student in the innovation level with a student just in the top 35% of those not involved.

  • Especially good for low achievers. Formative assessment helps all pupils but "gives particularly good results with low achievers" when it targets specific problems and gives achievable targets.

  • Marks alone do not help. If pupils get only marks or grades, they do not benefit from the feedback. Feedback improved learning when it gave specific guidance on strengths and weaknesses, preferably without an overall mark.

  • Questions need thinking time. Teachers often answered their own question after only two or three seconds, so mostly quick factual questions got asked and many pupils stopped trying to think.

They also argued that self-assessment by pupils is essential, not a luxury. Drawing on Sadler, they describe feedback as having three parts: the goal, evidence about where you are now, and a way to close the gap.

What did later researchers say?

The headline numbers did not hold up as well as the practices. Randy Bennett (2011) argued that "formative assessment" is not yet a well-defined set of practices, so effects should vary widely, and that commonly quoted effect sizes come from sources that are untraceable, flawed, dated or unpublished.

Kingston and Nash (2011) reviewed more than 300 studies and found only 13 with enough information to calculate effect sizes. The weighted mean was 0.20, smaller than the 0.4 to 0.7 often quoted. The estimates were lower for maths (0.17) and science (0.09) than for English language arts (0.32). Approaches based on teacher professional development and computer-based formative systems did somewhat better.

The fair reading: using evidence of learning to decide what to do next is sound, but how it is done matters a great deal, and the average gain is modest.

A three-question feedback loop for your own tests

You can run formative assessment on yourself after every chapter test or mock, using the three parts above.

  1. Where am I going? Write the goal in specific terms: "solve rotational dynamics problems with a rolling constraint", not "get better at physics".

  2. Where am I now? For each wrong answer, write one line on what went wrong: concept, method, calculation or time. Ignore the total score for now.

  3. How do I close the gap? For each error type, write one action for this week, such as "redo three rolling problems without notes on Wednesday".

Here is an example note after a mock:

  • Q7, rolling ball: used v = ωr when the ball was slipping. Concept error. Action: reread the rolling-without-slipping condition, then solve two slipping problems.

  • Q12, capacitor network: added series capacitors like resistors. Method error. Action: make a two-line comparison card for series and parallel rules.

  • Q19, ran out of time. Action: practise one timed section on Saturday.

For parents: Black and Wiliam's point about marks applies at home. Ask "what was the hardest question and what would you do differently?" before "what did you get?" Our guide to talking to teenagers about marks has more phrases.

For a full post-test routine, see how to analyse a mock test step by step. If you use AI to check your work, read what good feedback on step-by-step solutions looks like.

For JEE Main and Advanced, Fermi Testprep pairs adaptive mocks with a study plan that updates based on mastery, so your next step can follow from what your last test showed.

About the author

More from Fermi

More in Great Minds