By Sagi Shaier · 23 October 2026 · 7 min read

A short history of NLP: from ELIZA to deep learning

The four eras NLP went through on its way to today's large models, why each shift happened, and what the 1960s chatbot ELIZA shows about sounding smart without understanding anything.

NLP (natural language processing) is the field of getting computers to work with human language, and its history is a steady move from rules people wrote by hand to patterns a machine learns from data. Each move happened because the older approach could not cope with how varied real language is. Knowing why rules gave way to statistics, and statistics to neural networks, makes it easier to judge what today's models do well and where people still read too much into them.

The four eras of NLP

Early NLP systems ran on rules written by hand. Then the field shifted to learning patterns from data with statistics. Then came neural networks, and after them the large pretrained models used today.

The first break came because hand-written rules could not keep up with how varied and messy language is. The later moves came from something else: learning richer patterns from more text with more compute.

ELIZA, one of the first chatbots

ELIZA was written in the mid 1960s and was one of the first programs that held a conversation in text. Its most famous script imitated a certain style of therapist, the kind who mostly reflects your statements back at you as questions.

You typed something and ELIZA replied in a way that felt like it was listening. It ran on very little computing power, and the effect on people was still surprising. Some users opened up to it.

ELIZA is worth studying because it seemed smart while doing something very simple.

How ELIZA worked: pattern matching

ELIZA worked by pattern matching. A pattern is a template with a gap in it, such as "I feel X", where X can be any words.

For example, take the input "I feel nervous about my exam." ELIZA matches the pattern "I feel X", so X is "nervous about my exam". It then swaps the pronouns, so "my" becomes "your". Finally it fills a reply template, "Why do you feel X?", and returns "Why do you feel nervous about your exam?"

Some rules only look for a keyword. If you typed "my mother hates me", ELIZA might notice "mother" and reply "Tell me more about your family."

Here is a tiny ELIZA in Python with three rules. Each rule is a regular expression (a text pattern) and a reply template, and the last rule catches anything the others miss.

1import re
2 
3# Pronoun swaps applied to the captured words
4SWAPS = {"my": "your", "i": "you", "me": "you", "am": "are"}
5 
6# Each rule: a pattern to match and a reply template
7RULES = [
8    (r"i feel (.*)", "Why do you feel {0}?"),
9    (r".*\bmother\b.*", "Tell me more about your family."),
10    (r"(.*)", "Please go on."),
11]
12 
13def reflect(text):
14    return " ".join(SWAPS.get(word, word) for word in text.split())
15 
16def eliza(sentence):
17    sentence = sentence.lower().strip(".!?")
18    for pattern, template in RULES:
19        match = re.fullmatch(pattern, sentence)
20        if match:
21            return template.format(*[reflect(g) for g in match.groups()])
22 
23print(eliza("I feel nervous about my exam."))
24print(eliza("My mother hates me."))
25print(eliza("The weather is nice today."))

It prints:

1Why do you feel nervous about your exam?
2Tell me more about your family.
3Please go on.

That is close to all ELIZA did. It had no memory of the conversation, no model of your life and no representation of what the words meant. It matched text patterns, swapped a few pronouns, filled a canned template and returned a reply that sounded relevant.

The ELIZA effect

People knew they were talking to a program, and many still felt it understood them. Some wanted privacy for their conversations with it.

The tendency to read real understanding and empathy into a system that has none is called the ELIZA effect. It is still around today. When a chatbot phrases something warmly, people are quick to assume there is a mind behind it.

ELIZA showed that matching words on the surface of a sentence and rearranging them can imitate conversation without any grasp of what the words mean.

Does convincing behavior prove understanding?

ELIZA connects to an older question. The Turing test reframed "can machines think?" as an imitation game. A judge chats by text with a person and with a machine, and the test asks whether the judge can consistently tell which is which.

ELIZA shows why that standard is complicated. It fooled people for a while without understanding the conversation. So a system can produce useful words for reasons much simpler than understanding, and convincing behavior on its own proves less than it seems to. Today's models are far more capable than ELIZA, and they still raise the same question about how much behavior alone can show.

Era one: hand-written rules

The first era of NLP was rule-based, and ELIZA belonged to it. People wrote grammars, dictionaries and transformation rules by hand. A typical rule says that if a sentence matches a certain pattern or contains a certain word, the system gives a certain response.

Rules can work well on narrow, tidy problems. The trouble is coverage. Natural language is full of exceptions, variation and ambiguity, and people invent new ways of saying things all the time. Every exception needs another rule. As the rule set grows, the rules start interacting in ways that are hard to maintain, so hand-written rules alone cannot cover open-ended language.

Era two: statistical NLP

The second era was statistical NLP. Instead of writing every rule by hand, systems learned probabilities from a corpus, a large collection of real text.

An n-gram is a run of n words in a row, and a bigram is a run of two. An n-gram model counts which word sequences appear often and uses those counts to guess the next word. In the picture below, the word "strong" is followed by another word 240 times in the corpus, and 152 of those times it is "coffee". So the model estimates the probability of "coffee" after "strong" as $152/240$, about 0.63, and guesses "coffee".

The same counting fits in a few lines of Python on a six-sentence corpus:

1from collections import Counter
2 
3corpus = [
4    "i need strong coffee",
5    "she drinks strong coffee every morning",
6    "strong coffee helps me focus",
7    "he made the coffee strong",
8    "strong tea is fine too",
9    "a powerful engine",
10]
11 
12# Count every pair of neighbouring words (a bigram)
13pairs = Counter()
14for sentence in corpus:
15    words = sentence.split()
16    pairs.update(zip(words, words[1:]))
17 
18after_strong = {w2: n for (w1, w2), n in pairs.items() if w1 == "strong"}
19total = sum(after_strong.values())
20for word, count in sorted(after_strong.items(), key=lambda kv: -kv[1]):
21    print(f"P({word} | strong) = {count}/{total} = {count / total:.2f}")
22 
23print("strong coffee:", pairs[("strong", "coffee")])
24print("powerful coffee:", pairs[("powerful", "coffee")])

1P(coffee | strong) = 3/4 = 0.75
2P(tea | strong) = 1/4 = 0.25
3strong coffee: 3
4powerful coffee: 0

Nobody wrote a rule saying "strong coffee" sounds more natural than "powerful coffee". The counts carry that preference, and they would carry it for thousands of other word pairs nobody thought to write down.

This approach became useful for two reasons. It handles uncertainty with probabilities instead of breaking on an input no rule covers, and it can improve as more text is added.

Era three: neural networks

The third era brought neural networks and deep learning. Instead of counting exact word sequences, these models learn a dense numeric representation of each word: a list of numbers the network picks during training.

Older systems leaned on features an engineer wrote by hand. Examples are "the word ends in -ing", "the word is capitalised" and "the word appears after the". A neural network figures out which features are useful on its own, so an engineer no longer has to list them.

Era four: transformers and large pretrained models

The fourth era is the current one: transformers and large pretrained models. A pretrained model is trained once on an enormous amount of raw text and then adapted to specific tasks such as classifying, translating or answering questions.

Seen as a whole, the history has one direction. Each era let the machine learn more of what engineers used to write by hand.

What the history means for NLP work today

Hand-written rules still have a place. A narrow, tidy problem such as pulling dates out of a fixed form can be handled by a few patterns, and those patterns are easy to read and fix. Once the input is open-ended text from real people, the number of exceptions usually grows faster than anyone can write rules for them, and a learned model is the usual choice.

A common mistake is judging a system by how fluent it sounds. The ELIZA effect works on experienced people too, so a smooth, warm reply is weak evidence that a system handled the input correctly. Check its answers on real examples where you know what the right output is.

Related concepts

The history sits inside a bigger question of what NLP is and which tasks it covers. What is NLP? covers the main tasks, why language is hard for a computer and why modern NLP learns from data. For the full path from text to large language models, see the NLP roadmap.

QuiddityML teaches this history at the start of its Intro to NLP track, with ELIZA as its own concept, and the exercises on it include putting ELIZA's reply steps in order and placing the four eras of NLP on a timeline.