The Wonder
How Can a Computer Speak Like a Human?
You type a few words into a chat box. Seconds later, a machine responds—not with a canned phrase, but with something that feels genuinely thoughtful. It understands context, nuance, even humor.
How is this even possible?
The Impossible Machine
For decades, this was science fiction. Computers could calculate, store data, and follow precise instructions—but they couldn't truly understand language. Every attempt to build a "thinking machine" hit the same wall: human language is messy, ambiguous, and deeply dependent on context that seems impossible to specify in code.
Consider a simple sentence: "I saw her duck."
Did someone observe a woman's waterfowl? Or did they witness her quickly lower her head? Humans resolve this ambiguity effortlessly, drawing on context, world knowledge, and intuition developed over a lifetime. How could a machine ever do the same?
Hold on to that sentence. The answer, when it finally came, looked nothing like anyone expected.
Two Ways to Build a Mind
Since the 1950s, as Geoffrey Hinton tells it, there have been two ideas about what intelligence really is.
The first says intelligence is reasoning: you write down symbols and rules for manipulating them, and thought is logic. For language, that meant a dictionary and a grammar— meaning captured as a web of relations between words. Decades of careful work went into it, and still no machine could read "I saw her duck."
The second says intelligence is learning: not rules handed down from above, but the strengths of connections in a network of simple units, slowly tuned by exposure to the world. Reasoning, on this view, can wait. First the system learns.
That second path carried a different idea of meaning, borrowed from psychology: the meaning of a word is a bundle of features. "Dog" is animate, is a predator, is a companion—a cloud of properties rather than a dictionary entry. The trouble was, nobody could say where the features should come from.
Hinton's answer, worked out in a tiny model in 1985 and scaled a billion-fold since: let the network learn the features itself, and learn how the features of one word should interact to predict the next. Relations are never stored—they're generated, on the spot, from features pressing against features.
How Words Shake Hands
Here is Hinton's way of picturing what that looks like inside the model.
A word is not a fixed symbol with a single meaning waiting in a dictionary. It is a shape in a space of thousands of dimensions—and a flexible shape, with hands and gloves held out on movable arms.
To read a sentence, the model lets these shapes deform. Each word reaches its hands toward the gloves of the words around it, bending a little until they fit. When every hand has found a glove and the whole structure holds together, that settling is the understanding of the sentence. No rule was consulted. The words simply found a way to fit.
Now return to "I saw her duck." On its own, "duck" holds both shapes at once—the bird and the crouch. But "saw" and "her" reach toward it, and only one arrangement lets every hand meet a glove. The rest of the sentence deforms "duck" into place. The ambiguity that stumped decades of hand-written rules dissolves in the fit.
This is the real answer to the wonder we started with. Not a faster dictionary, not statistics pasted together, but meaning found the way you find it—by letting words settle into a shape that holds.
The Mirror
This is where the story turns from engineering into something closer to a mirror.
You might expect the model to understand in some alien, mechanical way—real understanding over here in human heads, a clever imitation over there in the machine. Hinton refuses the split. In his Romanes Lecture at Oxford, he points out that the best theory we have of how you understand a sentence is the same theory: features, and the interactions between them. His 1985 model was built, first of all, as a model of people.
Look closer, and the resemblance has an engine. A language model is built by one relentless exercise: predict the next word, see what actually came, and adjust to be a little less wrong next time. The neuroscientist Karl Friston argues that a brain runs on the same loop, one level up. In his Free Energy Principle, the brain is a prediction machine: it carries a model of the world, forecasts what will happen next, and works ceaselessly to minimize surprise—the gap between what it expected and what its senses report. It closes that gap two ways: update the model, or act on the world to make it match the prediction.
One predicts the next word; the other predicts the next moment. The substrate could hardly be more different. The drive is the same.
The resemblance runs further than we might like. Language models confabulate—they state things that sound right and aren't. So do we. Ask anyone to recall a meeting from years ago and they'll invent details, misattribute quotes, and feel certain about all of it. There is no clean line, in us, between a true memory and a made-up one. In this, the machine is not failing to be human. It is being uncomfortably like one.
Why This Matters
We've created something genuinely new—not a faster calculator or a better search engine, but a different kind of tool. One that can:
- Explain complex topics in simple terms
- Help write and revise text
- Analyze and summarize documents
- Generate code from natural language descriptions
- Engage in nuanced conversations
This isn't magic, but it's also not fully understood. Even the researchers who build these systems are sometimes surprised by what they can do—and the deeper surprise is what they reveal about us. To explain the machine, we reached for the best account we have of our own minds, and found it fit. That mystery—what these systems truly are, and how much of ourselves we see in them—is part of what makes this moment in history so fascinating.
Key Takeaways
- Two old ideas of meaning—words as relations, words as features—were unified by learning the features from data
- A word's meaning behaves like a flexible, high-dimensional shape; understanding a sentence is those shapes deforming until they fit
- Hinton's claim: the features and their interactions don't produce understanding—they are it
- The same theory is our best model of how humans understand, blurring the line between mind and machine
