On this page
In July 1958, the US Office of Naval Research unveiled a program by a young psychologist, Frank Rosenblatt, that got better at a task by being corrected. He called it the perceptron: a simple model of a brain cell, and an ancestor of today's neural networks.
What was Frank Rosenblatt's perceptron?
Rosenblatt set out the idea in a 1958 paper, "The perceptron: A probabilistic model for information storage and organization in the brain", in Psychological Review. He was then a research psychologist at the Cornell Aeronautical Laboratory in Buffalo, New York, according to the Cornell Chronicle.
The perceptron was a single-layer neural network: an algorithm that sorts each input into one of two categories, as the Cornell Chronicle describes it. At the July 1958 demonstration, an IBM 704 computer was fed punch cards, and after 50 trials it taught itself to tell cards marked on the left from cards marked on the right. The New York Times headline read "New Navy Device Learns by Doing". Rosenblatt called it "the first machine which is capable of having an original idea".
The hardware version, the Mark I Perceptron, was a room-size grid of 400 light-registering sensors designed to perceive and sort images, Smithsonian magazine reports. The original is now in the Smithsonian's collections.
How did the perceptron learn?
A perceptron takes some inputs (for example, which light sensors are lit), multiplies each by a weight, adds them up with a bias term, and answers 1 if the total is above zero, 0 if not.
Learning means changing those weights. As philosopher Danielle Williams explains in The Conversation, after a wrong answer the machine would alter its connections so that it could make a better prediction. In the textbook rule, nothing changes when the answer is right. When it is wrong, you add (target minus output) times each input to that input's weight, and (target minus output) to the bias.
Can you train a perceptron by hand?
Yes. Take a card with two squares, like the 1958 left-or-right cards. Let x₁ = 1 if the left square is marked and x₂ = 1 if the right one is. We want output 1 for "left" and 0 for "right". Start with every weight at zero: w₁ = 0, w₂ = 0, b = 0.
Show a left card (x₁ = 1, x₂ = 0). The total is 0, so the output is 0. Wrong, because the target is 1. Add 1 to w₁ and b: now w₁ = 1, w₂ = 0, b = 1.
Show a right card (x₁ = 0, x₂ = 1). The total is 0 + 0 + 1 = 1, so the output is 1. Wrong, because the target is 0. Subtract 1 from w₂ and b: now w₁ = 1, w₂ = -1, b = 0.
Show the left card again. The total is 1, so the output is 1. Correct, so nothing changes.
Show the right card again. The total is -1, so the output is 0. Correct. The machine has learned the task.
Nobody typed in "look at the left square"; two corrections set the weights. The diagram above runs the same rule on the logic AND task: each dashed line is the dividing line after a correction, and the gold line is where it stops.
Why did the perceptron fall out of favour?
In 1969 Marvin Minsky and Seymour Papert published the book Perceptrons, which attacked Rosenblatt's work, the Cornell Chronicle notes. Rosenblatt drowned in a sailing accident in 1971, on his 43rd birthday.
A few lines of algebra show the main weakness. Try to make a perceptron output 1 when exactly one square is marked (a task called XOR):
Blank card: we need b ≤ 0.
Left card: we need w₁ + b > 0.
Right card: we need w₂ + b > 0.
Add the last two lines: w₁ + w₂ + b > -b, which is at least 0. So a card with both squares marked also gives output 1, which is wrong.
No weights work. A single perceptron can only split its inputs with one straight line, and XOR cannot be split that way.
How is the perceptron related to today's neural networks?
Later researchers got around this limit by stacking perceptron-like units in layers and training them together, the idea behind today's deep networks. The Cornell Chronicle headline says the perceptron "paved the way for AI – 60 years too soon", and in 2004 the IEEE established the IEEE Frank Rosenblatt Award.
For context, read how John McCarthy named the field of artificial intelligence, how Alan Turing defined what a computer can do and how Golgi and Ramón y Cajal revealed the neuron.
If weighted sums or straight-line boundaries feel shaky, the Fermi Concept Library has a lesson for every concept in mathematics and computing. For ideas easier to see than to read, the Fermi Simulation Lab has interactive simulations.
Pick up a pencil, run the AND task, and you will have done by hand what Rosenblatt's machine did in 1958.
