Affordances

In 1979, the psychologist James J. Gibson published a book that quietly upended centuries of thinking about perception. The key move was a single question: instead of asking how the brain constructs a picture of the world from raw sensory data, Gibson asked what information the environment already provides.

His answer introduced a word that would escape psychology and colonize design, robotics, philosophy, game theory, and AI: affordance.

"The affordances of the environment are what it offers the animal, what it provides or furnishes, either for good or ill." A flat, rigid, knee-high surface affords sitting. A graspable object affords picking up. A cliff edge affords falling off. These are not interpretations imposed by the mind — they are real properties of the organism-environment relationship.

The word itself was coined from the verb "to afford" — what the environment affords to a particular creature with a particular body and particular needs. A surface that affords walking for a human does not afford walking for a fish. The affordance belongs to neither the surface alone nor the animal alone. It exists in the relation between them.

Neither Objective nor Subjective

Here is where Gibson gets genuinely radical. He writes: "An affordance is neither an objective property nor a subjective experience; or it is both if you like. An affordance cuts across the dichotomy of subjective-objective."

This is not vagueness. It is a deliberate rejection of the framework that has organized Western philosophy since Descartes: the split between the world "out there" (objective, measurable, mind-independent) and experience "in here" (subjective, private, constructed).

Consider a cliff edge. Its height is an objective fact — 30 meters, measured by instruments, the same for everyone. But its affordance — the falling-off-ability — is not simply objective. It depends on the animal. For a mountain goat, that cliff affords a walkway. For a human, it affords danger. For a bird, it affords a launching point. The affordance is real, not imagined — but it only exists relative to a particular kind of body.

This is why affordances mattered beyond psychology. They suggested that the most important features of reality aren't the ones physics measures (mass, wavelength, molecular composition) but the ones that matter for living: can I walk on this? Can I eat this? Will this support me? The world, seen ecologically, is not made of matter — it is made of possibilities for action.

Direct Perception

The standard model of perception, inherited from the British empiricists and still dominant in much of cognitive science, goes like this: raw sensory data arrives at the eyes (or ears, or skin), gets processed by the brain, and gets assembled into a meaningful picture of the world. Perception is computation on impoverished inputs.

Gibson rejected this entirely. He argued that the information available to a moving, exploring organism is far richer than the snapshot model assumes. Consider optic flow — the streaming pattern of visual change as you move through the world. When you walk forward, the visual field expands outward from a central point. When you turn, the whole field shifts. This flow specifies your movement directly, without the brain needing to calculate it.

This is direct perception: the claim that organisms perceive affordances without needing to construct them through inference. The information is already structured in the ambient array of light, sound, and touch. The animal's job is to pick it up, not to compute it.

The implications are sweeping. If perception is direct, then representation — the dominant concept in cognitive science and AI — is not the foundation of mind. The world is not re-built inside the head. It is encountered, explored, and acted upon.

From Ecology to Design

Gibson died in 1979, the same year his masterwork was published. He might have been surprised to learn that the concept he developed for ecological psychology would become one of the most influential ideas in design.

The bridge was Don Norman, who encountered Gibson's work and brought "affordance" into The Design of Everyday Things (1988). But Norman adapted the concept — and the adaptation matters.

The Phenomenology Connection

Gibson developed his theory independently of European phenomenology, but the convergence is striking. Two traditions, working from opposite ends, arrived at the same insight: we perceive the world primarily as a field of possibilities for action, not as a collection of neutral properties.

Heidegger's concept of Zuhandenheit (readiness-to-hand) describes exactly this. The hammer, when you're hammering, isn't perceived as "a wooden handle attached to a metal head of certain weight." It disappears into the activity. You perceive the nail going in, the board being joined. The hammer is ready-to-hand — transparent in use, encountered as an affordance rather than as an object with properties.

Merleau-Ponty went further, arguing that the body itself is not an object in the world but the medium through which we have a world at all. The body-subject — his term — doesn't represent space; it inhabits it. Your body "knows" the width of a doorway not through measurement but through a lived sense of your own dimensions. This is affordance perception at its most embodied.

Evan Thompson and the enactivist tradition have taken this further, proposing that cognition itself is not representation but enaction — the ongoing coupling between organism and environment. On this view, affordances aren't something the mind detects; they are what cognition fundamentally is.

Affordances and AI

In the 1980s, Rodney Brooks at MIT made a Gibsonian move in robotics. His behavior-based robots had no internal model of the world. They responded directly to environmental features — light gradients, surface edges, obstacle proximity — using simple sensorimotor loops. His slogan: "The world is its own best model."

Brooks' robots could navigate cluttered rooms and avoid obstacles, while the "traditional" AI robots of the era — loaded with symbolic world models — got stuck trying to update their representations. The lesson: for embodied action in the real world, detecting affordances directly often beats constructing and maintaining internal maps.

The affordance concept has resurfaced in modern AI through projects like Google's SayCan, which grounds large language models in a robot's actual capabilities. The LLM proposes actions ("pick up the sponge"), but a separate affordance model checks: can this robot actually do that, right now, in this environment? The language model provides semantic knowledge; the affordance model provides ecological reality.

This connects directly to the broader question of AI alignment. If AI systems don't perceive affordances the way embodied creatures do, their "understanding" of what actions are appropriate in a situation may be fundamentally different from ours — even when their language about it sounds identical.

Affordances Everywhere

Once you see affordances, you see them everywhere. The concept has proven remarkably portable across disciplines — perhaps because it captures something genuinely fundamental about how living things relate to their environments.

The Open Question

Can a disembodied system — an LLM, a planning algorithm, a neural network trained on text and images — genuinely perceive affordances? Or can it only describe them, borrowing the words of embodied beings who actually encounter the world through bodies?

This is not a trivial question. It connects to the deepest puzzles in philosophy of mind. The Chinese Room argument asks whether symbol manipulation alone can constitute understanding. Affordance theory sharpens the question: understanding, for Gibson, is inseparable from the capacity to act. If you can't walk on a surface, you can't understand what "walkable" means in the way a walker does.

Andy Clark and the extended mind tradition suggest a more generous answer: perhaps cognition isn't limited to biological bodies. If a system is sufficiently coupled with its environment — sensing, acting, adapting — it might develop something genuinely analogous to affordance perception, even without flesh. The question then becomes not "does it have a body?" but "is it ecologically embedded?"

Current AI systems are, for the most part, not ecologically embedded. They process inputs and produce outputs, but they don't inhabit an environment in Gibson's sense. The question of whether they could — and what would change if they did — remains one of the most fertile open questions at the intersection of AI and philosophy.

Explore Further