On this page
A computer does not know what a cat is. It learns from thousands of pictures that a person has already marked "cat" or "dog". Chatbots learn good answers from bad in a similar way, from replies that people wrote or ranked. Here is who does that work, and a 40-minute activity to try it in class.
Who labels data for AI training?
Crowdworkers. To build the ImageNet photo collection, its creators used Amazon Mechanical Turk, "an online platform on which one can put up tasks for users for a monetary reward", according to the ImageNet challenge paper by Russakovsky and colleagues.
Hired contractors. For InstructGPT, a language model trained to follow instructions, OpenAI "hired a team of about 40 contractors on Upwork and through ScaleAI", the InstructGPT paper says. Most of them spoke English and lived in the United States or Southeast Asia.
Data companies, including in India. Karya, a Bengaluru-based startup, splits data jobs into small paid tasks that rural workers do on its app, collecting audio data in their native languages, such as Kannada, Marathi, Telugu and Odia. Karya sells that data to tech companies, according to a NITI Aayog Frontier Tech case study, which lists "35,000+ people impacted".
How does a labeller teach AI to tell a cat from a dog?
The ImageNet team showed workers a set of candidate photos with the definition of one category, plus a link to Wikipedia. Workers then marked which photos really showed that thing, as the ImageNet challenge paper describes.
People make mistakes and disagree, so several people labelled each image independently. A photo counted only if it won "a convincing majority of the votes". Harder categories needed more votes: the authors note that five people might be needed to agree on a Burmese cat, but far fewer for an ordinary cat. To set the rule for each category, they first asked at least 10 people to vote on a sample of its images.
When an independent group checked the images in 80 sampled ImageNet categories, 99.7% of them on average really showed their category. In 2012 the challenge swapped 90 of its 1,000 categories for dog breeds, so a model had to tell one breed from another, not just a dog from a cat.
How do people teach a chatbot good answers from bad?
InstructGPT was trained in two human steps, according to Ouyang and colleagues' paper. First, labellers wrote example answers showing the behaviour they wanted. Then they ranked several of the model's replies to the same prompt from best to worst, and the model was trained to prefer the highly ranked kind.
In the paper's human evaluations, people preferred the outputs of a 1.3-billion-parameter InstructGPT model over the 175-billion-parameter GPT-3, a model more than 100 times bigger. For the full pipeline, read our guide to how AI chatbots are trained with human feedback.
What happens when the labels are wrong?
The model learns the mistake. A NeurIPS 2021 study by Northcutt, Athalye and Mueller estimated that at least 3.3% of labels were wrong, on average, across the test sets of 10 widely used datasets. For the ImageNet validation set the figure was at least 6%. Wrong test labels also distort scores, so a model can look better or worse than it is. Our explainer on accuracy, precision and recall shows how those scores are calculated.
Try it: a 40-minute classroom labelling activity
You need 20 numbered pictures, slips of paper and a board. Mix easy cats and dogs with tricky ones: a fox, a cartoon dog, a blurry photo.
Write the rule (5 minutes). As a class, write a label card: "CAT if…, DOG if…, UNSURE otherwise." This is your category definition, like the one ImageNet workers saw.
Label alone (10 minutes). In groups of three, each student labels every picture on their own slip. No talking.
Tally (5 minutes). For each picture, the group writes 3-0, 2-1 or a three-way split, and the majority label.
Argue the hard ones (10 minutes). Put every 2-1 picture on the board. Ask: "What made this one hard? Was it the photo, or our rule?" Rewrite the rule and relabel those pictures.
Rank answers (5 minutes). Show a question, such as "Why does the Moon have phases?", with two short written answers. Each student picks the better one and writes one reason. Compare reasons; this is the ranking step chatbot labellers do.
Plant an error (5 minutes). Secretly mark one dog as "cat" in the answer key and reveal it. Ask: "If a model learned from this key, what would it get wrong, and how would we catch it?"
To extend it, students can train a model on their own labels with our no-code image classifier school project.
The tally raises a real maths question: why is a vote of three more reliable than one person? If a student wants to work through the probability, Fermi's AI tutor can work out what they already know and build a plan for the session.
Next time an app names a breed from a photo, remember that somewhere, a person said "dog" first.
