On this page
In 1966, an MIT professor named Joseph Weizenbaum created a computer program called ELIZA to study natural language. It used a script called DOCTOR to mimic a Rogerian psychotherapist, reflecting users' statements back as questions. He intended to show how superficial human-computer conversation really was, but instead, the program fooled its own users. People began confiding their deepest secrets to the simple machine, a phenomenon that changed Weizenbaum from an AI pioneer into one of its sharpest critics.
How did Weizenbaum build the first chatbot?
Joseph Weizenbaum was born in Berlin in 1923 and escaped to the United States in 1936. During the Second World War, he served as a meteorologist in the U.S. Army Air Corps. He later joined the Massachusetts Institute of Technology (MIT), where he explored how computers process lists of words.
In 1966, Weizenbaum published a computer program called ELIZA DOI: 10.1145/365153.365168. He built it to study natural language communication between humans and machines, and named it after Eliza Doolittle from the play *Pygmalion* because it could be taught to "speak" differently by loading new scripts.
The most famous script was DOCTOR, which mimicked a Rogerian psychotherapist. Weizenbaum chose this specific role because a Rogerian therapist often reflects a patient's statements back as questions. This allowed the program to maintain a conversation without needing a database of real-world knowledge.
ELIZA worked through a clever two-step process of pattern matching:
Decomposition: The program scanned what the user typed for specific keywords. If it found the word "mother", it broke the sentence down into fragments around that word.
Reassembly: It then applied a rule to flip pronouns—turning "my" into "your"—and slotted those fragments into a pre-written response. An input of "I am worried about my mother" became "Tell me more about your family."
If the program found no keywords, it printed a content-free remark like "Please go on", or recalled a previous keyword from its memory.
What was the ELIZA effect?
Weizenbaum published his work in the journal *Communications of the ACM*, intending to show his colleagues just how superficial human-computer conversation really was. Instead, the program fooled its own users.
People began confiding their deepest secrets to the simple machine. Weizenbaum's own secretary had watched him build the program and knew exactly how the pattern matching worked. Yet, when she sat down to test it, she quickly became engrossed in the conversation. After just a few exchanges, she turned to Weizenbaum and asked him to leave the room so she could talk to the machine in private (Wikipedia).
This tendency for people to project empathy, understanding and human intelligence onto a simple computer program became known as the ELIZA effect. Weizenbaum was stunned by how quickly ordinary people formed emotional bonds with a machine that understood nothing.
Why did Weizenbaum turn against artificial intelligence?
While the field of computer science had been full of optimism since John McCarthy, the Dartmouth proposal and the birth of AI in 1956, Weizenbaum became deeply disturbed by the public's reaction. What worried him most was when some psychiatrists suggested that ELIZA could be scaled up to serve as an inexpensive, automated therapy tool.
Weizenbaum became one of the sharpest critics of artificial intelligence. In his 1976 book, *Computer Power and Human Reason: From Judgment to Calculation*, he argued that there is a strict boundary between what a machine can calculate and what requires human judgment (Wikipedia).
He pointed out that computers lack human biology, emotional context and lived experience. Because of this, they can never truly understand a human problem. This explored a different boundary than Alan Turing's 1936 paper: what a computer can and cannot do. Turing proved the mathematical limits of machines; Weizenbaum questioned their moral limits.
He famously warned that there are certain tasks we should never hand over to computers, regardless of how advanced the programming becomes. Roles that require genuine human compassion, wisdom and values—such as a psychotherapist, a judge in a court of law, or a military commander—must remain strictly in human hands.
How can you test if a chatbot is teaching you?
Today, many students use AI to help with their studies. It is easy to fall for the ELIZA effect and assume a program understands your struggles just because it generates fluent, confident text.
Students often wonder, Can AI tutors teach better than chatbots that give answers? The answer lies in whether the tool is designed to mimic a conversation or actually build your understanding. When you sit down to study, you need a tool that does more than match patterns and print textbook definitions. Fermi's AI tutor is built differently. The tutor teaches rather than answers: it holds a line of reasoning across a session, notices where a student is stuck, and decides whether to explain, ask or show.
If you are using a standard chatbot for your homework, here is a simple checklist to see if it is actually teaching you, or just feeding you a script:
Make a deliberate small mistake: Early in your working out, write a wrong number. A basic pattern matcher will often ignore it and print the final correct answer anyway. A real tutor will pause and ask you to look at that specific step.
Ask "Why does this work?": If the chatbot pastes a textbook definition, it is matching keywords. If it builds an analogy based on the problem you are currently solving, it is holding the context of your session.
Say "I don't know where to start": An answer engine will usually just solve the problem for you immediately. A helpful tool will give you the first hint or ask a leading question to get you thinking.
By understanding the limits of what machines can do, as Weizenbaum warned in 1976, you can use technology to push your own thinking further, rather than letting a machine do the thinking for you.
