On this page
In April 1982, the physicist John Hopfield published a paper in PNAS, Neural networks and physical systems with emergent collective computational abilities. Its abstract describes a memory that "correctly yields an entire memory from any subpart of sufficient size." In 2024 that idea earned him a share of the Nobel Prize in Physics, with Geoffrey Hinton, "for foundational discoveries and inventions that enable machine learning with artificial neural networks".
What is a Hopfield network?
A Hopfield network is a set of nodes, each joined to every other, that stores patterns and recreates the most similar stored pattern when given an incomplete or distorted one, according to the Nobel Committee's popular explanation. Think of each node as one pixel in a black-and-white picture; the memories live in the connection strengths.
Hopfield borrowed the maths from magnetism. In a magnetic material, each atom's spin makes it a tiny magnet, and neighbouring spins influence each other, the physics behind the Ising model of magnetism.
The 1982 article used a tiny network: 30 nodes, all connected by 435 connections, and fewer than 500 parameters in total, according to the Nobel popular information.
How does a Hopfield network recall a memory from a clue?
The network gives every possible pattern an energy, calculated from all the node values and all the connection strengths, as the Nobel popular information explains. Storing a picture means setting the connections so that the picture has low energy.
Recall runs downhill. The network visits nodes one at a time, flipping any whose change lowers the energy, until no flip helps; by then it has often rebuilt the stored picture. Hopfield compared this to a ball rolling across peaks and valleys with friction: drop it somewhere and it rolls into the nearest valley and stops, the same page says. Each stored pattern is a valley.
Worked example: a four-node memory you can check by hand
We use +1 and −1 for the pixel values instead of 1 and 0; the arithmetic is cleaner and the idea is the same. A connection is positive when two nodes agree in the pattern and negative when they disagree.
Store the pattern A = (+1, −1, +1, −1). Set each connection to the product of its two nodes: w12 = −1, w13 = +1, w14 = −1, w23 = −1, w24 = +1, w34 = −1.
Define the energy as E = −(sum over all six pairs of w × s_i × s_j). For A, each of the six terms is +1, so E = −6. That is the bottom of the valley.
Give a damaged clue with node 3 wrong: (+1, −1, −1, −1). The six terms are now +1, −1, +1, −1, +1, −1, so E = 0. The clue sits on the slope.
Update node 3. Its input is w31 × s1 + w32 × s2 + w34 × s4 = (+1)(+1) + (−1)(−1) + (−1)(−1) = 3. The input is positive, so node 3 flips to +1.
The network is back at A, with energy −6, and no further flip lowers it. The memory is recalled.
One surprise: the mirror pattern (−1, +1, −1, +1) also has energy −6. Flipping every node leaves each product unchanged, so a Hopfield network stores every pattern together with its photographic negative.
How many memories can a Hopfield network store?
In 1985, Daniel Amit, Hanoch Gutfreund and Haim Sompolinsky showed in Physical Review Letters that, for very large networks storing random patterns, the model works as an associative memory only while the number of patterns stays below about 0.14 times the number of nodes. As a rough guide, that puts a 100-node network at around 14 patterns.
Hinton later used the Hopfield network as the foundation for the Boltzmann machine, the Nobel press release notes. Follow that thread in our piece on how backpropagation taught neural networks to learn.
What does a Hopfield network teach you about studying?
A Hopfield network is a model, not a map of your brain, but it is a useful picture. The Nobel popular information introduces Hopfield's work with someone hunting for a rarely used word, trying "ramp" and "radial" before landing on "rake", as an everyday version of searching memory from a partial cue.
For revision, the picture suggests three habits:
Practise from partial clues. Cover a derivation after its first line and rebuild the rest, the way the network rebuilds a pattern from a fragment.
Keep look-alike ideas apart. In the model, overcrowded memories stop working; when two formulas look similar, write down what makes each different.
Try another cue. A memory you cannot reach from one clue may come back from another, the idea behind Endel Tulving's work on why you forget answers you know.
Energy landscapes are easier to see than to read. The Fermi Simulation Lab has interactive simulations for ideas like that, and the Fermi Concept Library gives each concept its own lesson, so you can fill gaps such as spin or energy first.
Next time half a formula comes back from a single symbol, picture a ball rolling into its valley.
