On this page
Every machine learning model learns from examples, and every model is tempted to remember the examples instead of the pattern behind them. If you have seen a classmate ace a repeated mock and then struggle in the real exam, you already understand the problem.
What is overfitting in machine learning?
Overfitting means "creating a model that matches (memorizes) the training set so closely that the model fails to make correct predictions on new data", according to Google's Machine Learning Crash Course.
The opposite quality is generalisation: the same course says a model that generalises well "makes good predictions on new data". If the terms model and training are new to you, start with our post on AI vs machine learning vs deep learning, with one example each.
How is overfitting like memorising past paper answers?
Picture two students revising from five years of JEE Main papers. Student A memorises the answers; Student B works out why each method works. On those papers, both score close to full marks. On this year's paper, with new numbers and a twist, Student A has nothing to fall back on.
Student A has overfitted. Here is how the pieces map:
Past papers are the training data.
Your marks on those same papers are the training score.
This year's paper is new data, and your mark on it is the test score.
Only the test score shows whether you learnt the pattern or the examples.
What is a simple example of overfitting?
This one you can check by hand.Suppose the hidden rule is y = x, but our measurements are a little noisy. We collect four training points: (0, 0), (1, 2), (2, 1) and (3, 3). Two of them are off by 1 because of the noise.
Model 1 is the least-squares straight line, y = 0.8x + 0.3 (our post on linear regression with Class 11 maths shows how to get it). It misses the training points by 0.3, 0.9, 0.9 and 0.3, so its mean squared error on training data is 0.45.
Model 2 is the cubic y = x³ − 4.5x² + 5.5x. It passes through all four points, so its training error is 0.
Now test both on three new points that follow the real rule, at x = 0.5, 2.5 and 4:
At x = 0.5 the true value is 0.5. The line predicts 0.7; the cubic predicts 1.75.
At x = 2.5 the true value is 2.5. The line predicts 2.3; the cubic predicts 1.25.
At x = 4 the true value is 4. The line predicts 3.5; the cubic predicts 14.
On this test data the line's mean squared error is 0.11, while the cubic's is about 34.4. The cubic did not learn the rule; it learnt the noise, the same way Student A learnt the past-paper answers instead of the method.
The scikit-learn underfitting vs overfitting example shows the same thing at a larger scale: it fits polynomials of degree 1, 4 and 15 to 30 noisy samples of a cosine curve. Degree 15 learns the noise, while degree 1 is too simple to follow the curve, which is called underfitting. So the cure is not the simplest model; it is the model that does best on data it has not seen.
Why does test data matter?
Google's course splits data into a training set the model learns from, a validation set for checking it during training, and a test set for the final evaluation of the trained model, as its page on dividing the dataset explains. The same page warns that a good test set should have zero examples duplicated in the training set, and that test and validation sets "wear out" with repeated use: the more you use them to make decisions, the less they tell you about new data.
The same happens when you sit one mock again and again: by the fourth attempt, your score measures memory, not skill.
How do you avoid overfitting your own exam prep?
Hold out a test set. Keep the most recent year's paper untouched until you sit it once, timed.
Change one thing. After solving a past question, change a number or condition and solve it again.
Say why each step works. If you can only say the answer, you have stored the example, not the method.
Mix topics so the chapter heading does not give the method away; our interleaved practice plan shows how.
Track two scores, on seen papers and fresh ones. A big gap is your overfitting signal.
When a changed question breaks your method, the Fermi Problem Library has worked problems with a step-by-step solver that explains the reasoning rather than printing an answer, so you can see which step you had memorised. If you are not sure which topics need that work, Fermi's AI tutor works out what you already know and builds a plan for the session.
Tonight, take one past question you know by heart, change a single number, and solve it from scratch. That is your first test set.
