On this page
Class 12 maths already contains the idea behind how AI models learn. In the NCERT chapter Application of Derivatives, f′(x) tells you where a function rises, falls and reaches a local minimum. Gradient descent uses the same derivative, but walks downhill in small steps instead of solving f′(x) = 0 in one go.
What is gradient descent, in simple words?
Picture a ball on a foggy hillside. It cannot see the valley, but it can feel which way the ground tilts, so it rolls a little that way, feels again and repeats. That tilt is the derivative.
Google's Machine Learning Crash Course defines it: "Gradient descent is a mathematical technique that iteratively finds the weights and bias that produce the model with the lowest loss" (Google, Gradient descent).
For one variable, the method is one line:
new x = old x − η × f′(old x)
η (eta), the learning rate, sets the step size. The minus sign sends x the way the ground falls.
The mathematician Claude Lemaréchal notes that textbooks credit the method to Augustin Louis Cauchy, in a note to the Paris Academy of Sciences dated 18 October 1847, written to help compute the orbits of heavenly bodies (Lemaréchal, Documenta Mathematica, 2012).
How do you do gradient descent by hand with Class 12 calculus?
Take f(x) = x² − 6x + 10, so f′(x) = 2x − 6. Setting f′(x) = 0 gives x = 3, and f″(x) = 2 > 0, so the minimum is f(3) = 1. (Shaky on the sign of f′(x)? Revise increasing and decreasing functions.)
Now let the ball find it, starting at x = 0 with η = 0.25:
Step 1: x = 0 − 0.25 × (−6) = 1.5
Step 2: x = 1.5 − 0.25 × (−3) = 2.25
Step 3: x = 2.25 − 0.25 × (−1.5) = 2.625
f(x) falls from 10 to 3.25, 1.5625 and about 1.14. The distance from 3 goes 3, 1.5, 0.75, 0.375: it halves every step. Subtract 3 from both sides of the update to see why: (new x − 3) = (1 − 2η) × (old x − 3), and with η = 0.25 that factor is 0.5. After 9 steps, x is within 0.01 of the minimum.
Our guide to maxima, minima and rates of change covers that test step by step.
What happens if the learning rate is too big or too small?
The factor (1 − 2η) predicts every case for this function:
η = 0.05: factor 0.9. x goes 0, 0.3, 0.57, 0.813 and needs 55 steps to get within 0.01 of 3.
η = 1: factor −1. x jumps 0, 6, 0, 6 forever.
η = 1.1: factor −1.2. x goes 0, 6.6, −1.32, 8.184, flying further out each time.
Google's course describes the same trade-off: "If the learning rate is too low, the model can take a long time to converge," and if it is too high, "the model never converges, but instead bounces around the weights and bias that minimize the loss" (Google, Learning rate).
How do AI models learn the same way?
A model makes predictions using numbers called weights, and a loss function measures how wrong they are. The loss plays the role of f(x); the weights play the role of x. Google's course describes the loop: calculate the loss, move the weights slightly in the direction that reduces it, and repeat until it stops falling (Google, Gradient descent).
The difference is scale: with many weights, the slope becomes a gradient, one partial derivative per weight, but each step is still old value minus learning rate times slope.
Backpropagation computes those slopes for layered networks. In a 1986 Nature paper, David Rumelhart, Geoffrey Hinton and Ronald Williams described a procedure that "repeatedly adjusts the weights of the connections in the network so as to minimize a measure of the difference" between the network's actual and desired output (Nature, 1986). For chatbots, see how large language models work, explained for students.
Try it yourself: a 20-minute practice plan
Take f(x) = x² − 4x + 7. Solve f′(x) = 0 first: the target is x = 2.
Start at x = 5 with η = 0.25 and take four steps by hand. The distance to 2 should halve each time.
Predict what η = 1 and η = 1.2 will do from (1 − 2η), then check.
Try f(x) = x⁴ − 4x² (valleys at x = ±√2) with η = 0.05. Start at x = 0.5, then x = −0.5: the ball finds the valley on its own side.
Start at x = 0. Since f′(0) = 0, the ball sits still on a hilltop, which is why the second derivative check matters.
If a step stops making sense, Fermi's AI tutor can draw the figure and check your understanding as it goes. To revise derivatives, the Fermi Concept Library has a lesson for every concept in mathematics.
Next time you find f′(x) in a board paper, you are doing by hand what most AI models do when they learn.
