On this page
You have played this game before. Twenty Questions, or the dichotomous keys in your biology textbook: "Does it have feathers? Yes. Does it fly? No." A decision tree in machine learning is the same idea, with one difference that matters. You make up the questions from experience. The computer has to work out which questions to ask, and in what order, from a table of examples.
All you need is a pencil, paper and eight animals.
What is a decision tree in machine learning?
The scikit-learn documentation, the guide to a widely used Python library, defines decision trees as "a non-parametric supervised learning method used for classification and regression". Supervised means you give the computer examples whose answers you already know. Classification means the answer is a label, such as mammal, bird or fish.
The tree is built from three parts:
Questions (nodes), such as "Does it have feathers?"
Branches, one for each answer, here yes or no
Leaves, where the questions stop and the tree gives its label
J. Ross Quinlan's paper "Induction of decision trees", published in the journal Machine Learning in 1986, summarised an approach to building decision trees from examples and described one such system, ID3, in detail.
How do you build a decision tree on paper?
Write these eight animals on a page, with the answer key beside each:
Cow (mammal)
Dog (mammal)
Bat (mammal)
Sparrow (bird)
Peacock (bird)
Ostrich (bird)
Rohu (fish)
Catla (fish)
Now write four yes-no questions: Can it fly? Does it have feathers? Does it live in water? Does it have four legs? Answer each question for each animal. Then try to build a tree that sorts all eight into the right class using as few questions as possible.
Most people reach for "Can it fly?" first. Watch what happens: the bat ends up with the birds, and the ostrich ends up with the cows. A question that feels natural can still sort the animals badly. That is exactly the problem the computer has to solve.
How does a computer pick the best question?
The computer tries every question and measures how mixed the groups are afterwards. One common measure is Gini impurity, which the scikit-learn guide writes as the sum of p(1 − p) over the classes, where p is the share of each class in a group. In plain terms: a group that is all birds scores 0, and the more mixed the group, the higher the score.
The guide says the algorithm runs a "greedy exhaustive search" over all features and keeps the split with the lowest weighted impurity. Weighted means a big group counts for more than a small one. Here is that calculation for our eight animals. Before any question, 3 mammals, 3 birds and 2 fish give an impurity of 1 − (9 + 9 + 4)/64 ≈ 0.66.
Can it fly? Yes: bat, sparrow, peacock (0.44). No: the other five (0.64). Weighted score: 0.57
Four legs? Yes: cow, dog (0). No: the other six (0.61). Weighted score: 0.46
Lives in water? Yes: rohu, catla (0). No: three mammals, three birds (0.5). Weighted score: 0.375
Feathers? Yes: all three birds (0). No: three mammals, two fish (0.48). Weighted score: 0.30
Feathers wins, so it goes at the top. The computer then repeats the process on the "no" group of five animals. "Lives in water?" splits them perfectly into fish and mammals, with a score of 0, so the tree is finished after two questions. "Can it fly?", the question most people start with, scored worst of all.
Why can a decision tree go wrong?
A tree that keeps asking questions can end up memorising its examples instead of learning a rule. The scikit-learn guide calls this overfitting and lists fixes such as limiting the depth of the tree. Try it: add a penguin and a whale to your list and run them through your two-question tree. The penguin still lands with the birds, but the whale lives in water, so the tree calls it a fish. A model only knows the examples it was shown.
To see how this connects to AI chatbots, our explainer on how large language models work covers a very different kind of model. If your school offers it, our guide to the CBSE Artificial Intelligence subject explains what it covers, and our post on taxonomy and the living world shows the biology side of classification.
If an idea such as impurity or information gain still feels slippery, Fermi's AI tutor can work out what you already know and decide whether to explain, ask or show, and the Fermi Concept Library has lessons on concepts across mathematics and computing. Start with your eight animals, though. A tree you built and scored yourself will teach you more than one you only read about.
