TL;DR

Analogies usually help us understand abstract concepts rather than explain their underlying logic. I have therefore always considered the comparison between deep learning and biological neurons rather imprecise, influenced more by the naming than anything else. Reading this book made me realize that their underlying logic is much closer than I had imagined, and that neuroscience has influenced the history of deep learning more deeply than I thought.

Intelligence can be explained, and explained in layers—even along an evolutionary timeline. Each layer marks a major breakthrough in our evolutionary history.

  • Animals with bilateral symmetry gained greater mobility, but also needed better steering. They evolved neuronal control over movement and turning. Signals could be combined along positive/negative and active/subdued dimensions into four states, producing primitive emotions. These animals also had neuromodulators such as dopamine and serotonin. Later, they learned to use patterns in neural signals to predict actual events, acquiring conditioned reflexes.
  • Vertebrates improved dopamine’s mechanism, decoupling reward from reinforcement and enabling temporal-difference learning. Groups of neurons could also work together on complex pattern recognition. The hippocampus laid the foundation for a genuine world model.
  • Mammals evolved a structure capable of simulating anything: the neocortex. Its recognition-model/generative-model architecture produced a world model that could predict anything, enabling vicarious trial and error and counterfactual learning. The aPFC then appeared, predicting the neocortex’s own activity and controlling its search over strategies, making model-based reinforcement learning possible.
  • Primates’ political lives created a need to model other minds. The gPFC emerged to predict the aPFC’s operation, performing a kind of inverse reinforcement learning. This produced an understanding of self and others and enabled learning by observing others.
  • Humans acquired language. Experiences and neocortical simulations could now be shared, allowing knowledge to accumulate until it reached a singularity and produced who we are today.
Organisms New structures Abilities How they learn
LUCA
Animals Neurons Internal digestion, unconditioned reflexes
Bilaterians Brain Primitive emotions, conditioned reflexes From actual actions
Vertebrates Cortex, hippocampus TD learning, pattern recognition From actual actions
Mammals Neocortex, aPFC World models, model-based reinforcement learning From imagination
Primates gPFC Mentalizing: modeling others By observing others’ actual actions
Humans Language From others’ imagination

Primitive Intelligence

Before any of these breakthroughs, neurons had to appear to lay the groundwork.

According to current research, life originated at alkaline hydrothermal vents. The earliest organisms depended on ion concentration gradients for energy, and there was no obvious competition between them. But the emergence of chloroplasts and mitochondria, and with them photosynthesis and respiration, ended that peace: organisms that did not photosynthesize obtained their energy from the tasty insides of those that did.

Near the top of today’s biological classification is the kingdom, including plants, animals, and fungi. From the perspective of photosynthesis, plants do the work, while animals and fungi enjoy its products. Fungi and animals, however, differ greatly in how they feed:

  • Fungi digest their food externally.
    • They generally spread spores near dead plants, develop hairlike filaments, secrete enzymes, and absorb the released nutrients.
  • Animals digest their food internally.
    • They generally have a cavity in which prey can be trapped and slowly digested.
    • This requires the cavity to close quickly when prey is captured. That was the earliest role of neurons: sense prey inside, then signal the muscles to close the cavity.

Two mechanisms are central to how neurons work:

  • They need a way to encode the signals they transmit.
    • Individual neural impulses have the same strength. A stronger stimulus instead produces more frequent impulses.
    • For example, signals in the optic nerve have the same strength in dark and bright surroundings; in bright surroundings, impulses simply occur more often.
  • Neurons have some capacity to adapt.
    • The relationship between impulse frequency and stimulus intensity is nonlinear, resembling an activation function in deep learning. Adaptation to an environment can shift the range in which that function is most sensitive.
    • For example, the illuminance humans can perceive may vary by tens of thousands of times—a large dynamic range—without impulse frequency varying by anything like as much.

Breakthrough #1: Steering and the First Bilaterians

Steering

Most animals we see today have bilateral symmetry: left–right symmetry, with a distinct front and back. This improves mobility because the structures responsible for locomotion mainly need to move the animal forward and backward. But it also demands better steering, since the body must be turned in the right direction. Under this pressure, our ancestors evolved a simple navigation mechanism, forming a simple brain.

Modern nematodes can be viewed as resembling early bilaterians. Their brains contain only a little over three hundred neurons, yet they can steer toward food using smell. The mechanism is simple: some neurons detect the concentration of molecules released by food—the smell—and send positive signals to the muscles responsible for forward movement. Other neurons detect falling concentrations, fullness through internal sensation, or danger. They connect to muscles controlling turns and cause the animal to turn.

Emotions

The actual arrangement may be slightly more complex. Sensory neurons pass initial signals to a few key interneurons. Those signals can be evaluated along two dimensions: positive versus negative valence, and high versus low arousal. Combining them gives a four-quadrant coordinate system: primitive emotions.

For a nematode:

  • Positive valence, high arousal: slow movement and frequent turns, on detecting nearby food.
  • Positive valence, low arousal: remaining still, for example when full.
  • Negative valence, high arousal: fast movement and infrequent turns, when escaping.
  • Negative valence, low arousal: remaining still under prolonged stress.

These quadrants map readily onto human states:

  • Positive valence, high arousal: excitement and exhilaration.
  • Positive valence, low arousal: contentment and relaxation.
  • Negative valence, high arousal: anxiety and fear.
  • Negative valence, low arousal: depression and sadness.

What we call emotions may therefore have originated in these states that helped animals find food and escape.

Our emotions persist: sadness may last for hours or days. This also makes sense in nematodes. Food odors do not necessarily form a smooth gradient; a worm may smell food briefly and then lose the scent. A persistent emotion—positive valence and high arousal here—can keep it searching. Likewise, continuing to flee after a threat improves the chance of survival.

Both humans and nematodes regulate emotions through neuromodulators, the two most important here being dopamine and serotonin. In nematodes, dopamine neurons extend small projections from the head to detect food. On finding it, they release dopamine and produce a high-arousal state. Serotonin neurons monitor food in the throat; its presence causes serotonin release and a feeling of satiety, a low-arousal state.

Dopamine, then, drives the search for food by signaling that something good is nearby. Serotonin signals that something good is happening. In humans, seeing food or a sexually attractive person releases dopamine; eating delicious food, having an orgasm, or completing a to-do item releases serotonin.

In an emergency, animals release chemicals such as adrenaline, triggering fight or flight while suppressing functions not immediately needed, such as digestion, immunity, and reproduction. This plainly cannot continue indefinitely, so a counter-regulatory mechanism is needed to restore normality. Opioids are released at that point. They promote appetite, help restore immune and digestive function, and reduce pain by inhibiting neurons associated with negative emotions. This is why opioids are often used as painkillers.

Experiments on nematodes show that prolonged danger eventually leads them to give up after a period of high arousal and enter a low-arousal state. Natural selection has shown that the energy-intensive emergency response cannot be sustained indefinitely. If danger never goes away, conserving some energy may offer a better chance of survival. In humans, prolonged chronic stress similarly activates serotonin, shuts down arousal and motivation, and produces low arousal and learned helplessness: depression.

Prediction

Pavlov’s conditioning experiments reveal a primitive capacity for associative learning: dogs can learn an association between a sound and food.

Even animals with as few neurons as nematodes have this ability. Put one group in fresh water and another in salt water, without food, then transfer them into an environment containing a piece of salt. The worms previously in fresh water approach the salt; those previously in salt water do not. This ability is also confined to bilaterians. Sea anemones, for example, do not have it: pair light with an electric shock, and they still respond only to the shock, never to the light.

If the two signals are repeatedly presented without a connection—sounding the tone without supplying food, for instance—the associative response weakens. But if the connection later returns, the association recovers quickly. This is called reacquisition. It makes sense: two things may genuinely be related in nature, with the occasional failure to coincide being accidental. Reacquisition helps discount those accidents.

Associative learning faces a problem: external signals are numerous and continuous, so which one is actually related to the target event? In reinforcement learning this is the credit assignment problem. Animals face it too, and use several approaches:

  • Eligibility traces: only signals occurring within, say, one second before food appears are eligible.
  • Overshadowing: attend only to the strongest signal and ignore weaker ones.
  • Latent inhibition: suppress familiar stimuli, treating frequent events as background noise.
  • Blocking: track a single cue; once it becomes associated with the target, prevent other cues from acquiring that association.

This learning should occur through changes in the strength of synapses connecting neurons. A strong synapse lets a weak signal from an input neuron activate the output neuron; a weak synapse requires a much stronger signal. Synapses have various mechanisms for strengthening and weakening. A protein mechanism might detect whether the neurons on both sides fire at similar times and strengthen the connection if they do. Proposed by psychologist Donald O. Hebb in the 1940s, this is called Hebbian learning. Reality is somewhat more elaborate: molecular mechanisms can detect finer timing, such as the input neuron firing before the output neuron, forming a connection only then and thereby producing an eligibility-trace effect.

Breakthrough #2: Reinforcing and the First Vertebrates

TD Learning

Reinforcement learning in computing began in the 1950s, when Princeton doctoral student Marvin Minsky tried to make a machine navigate a maze. Training was simple: reward it for reaching the exit. The algorithm did not perform well in complex situations. After hundreds of steps, it was unclear which steps should count as good or bad; one reward for the entire sequence was insufficient. This was another credit assignment problem, but across time.

In his 1984 doctoral thesis, Richard Sutton proposed an algorithm for temporal credit assignment: replace actual rewards with predicted rewards. His actor–critic model had an actor taking actions and a critic evaluating how much reward the current action deserved. In checkers, for example, the critic might regard the first nine moves as leaving the position even, then assign a high reward to a brilliant tenth move that changed the situation. This is temporal-difference, or TD, learning. But Sutton’s logic was circular: the actor’s knowledge of what to do depended on the critic’s evaluations, while the critic’s estimate of the chance of winning from a board position depended on the actor’s future actions.

In 1989, IBM’s Tesauro happened upon Sutton’s idea and implemented it for backgammon by training actor and critic together. The result, TD-Gammon, was state of the art at the time. It demonstrated that reinforcement learning could work in machines, and subsequent applications achieved human-level performance on many tasks.

That proved its effectiveness in machines, but was it merely a clever algorithm, or a general pattern by which biological intelligence emerged? A deeper understanding of dopamine’s function suggests the latter.

In one experiment, monkeys were shown a geometric image, given sugar water after a delay, and monitored for dopamine-neuron activity. Initially, that activity peaked when they received the sugar water. After several repetitions, it peaked at the image instead, with little response to actually drinking. Dopamine thus provides a reward for expectation: anticipating something good produces dopamine. Later in training, the image was when the monkey acquired that expectation; the actual pleasure of drinking no longer changed it.

More interestingly, some details match the techniques used in machine reinforcement learning. If a monkey learns that sugar water will arrive in four seconds rather than sixteen, its dopamine neurons respond more strongly to the image. This is discounted reward: good things in the near future receive stronger reinforcement, while distant ones are discounted.

Dopamine, in other words, is not a reward signal but a reinforcement signal. It encourages the animal to repeat the behavior that brought about its present situation. As Sutton originally discovered, reinforcement and reward are decoupled, making effective training possible. This also explains why people can become addicted to dopamine rewards without actually deriving much pleasure.

The nematode’s earlier dopamine signal—“there is something good, food, nearby”—is similar to this more complex mechanism. It reinforces continued searching, so it is easy to imagine how dopamine evolved into the mechanism described here.

Pattern Recognition

Many signals are not sensed by a single receptor. Smelling a particular odor, for example, means olfactory neurons jointly detecting several kinds of airborne molecule. Recognition therefore requires a group of neurons to work together to detect a pattern: pattern recognition.

Animals such as nematodes are limited to sensing through individual neurons; pattern recognition is unique to vertebrates.

Pattern recognition is difficult. Delicious prey and a dangerous predator may have odors that overlap in many respects—in many of their constituent molecules. In fish, olfactory neurons send their output to a structure on top of the brain called the cortex. Its pyramidal neurons have hundreds of dendrites for input and thousands of synapses for output.

The connections from olfactory input to cortex have an interesting organization, which we can describe as expansion and sparsity:

  • Expansion increases dimensionality: fewer olfactory neurons connect to more cortical neurons.
  • Sparsity means they are not fully connected: a given olfactory neuron connects only to some cortical neurons.

This arrangement may help solve the discrimination problem. Although predator and food odors overlap, they activate different cortical neurons. This is called orthogonalization.

The system also needs to generalize: two encounters with an odor may be similar but not identical. When an odor pattern activates a group of cortical neurons, Hebbian plasticity connects that group, forming an autoassociation. Later, activating only part of the pattern can activate the whole. The animal has acquired a way to address a memory through a subset of its contents.

Curiosity

Reinforcement learning unites two opposing processes:

  • Reinforce a behavior, encouraging the same behavior next time.
  • Explore new behaviors, making it possible to discover good ones.

In machine learning, this tension can partly be handled by temperature-scaling the actor’s outputs and choosing an action randomly. When that is insufficient, a mechanism such as the “curiosity” proposed by DeepMind in 2018 provides an intrinsic reward for trying something new.

DeepMind drew inspiration from animal curiosity. Since TD learning is found only in vertebrates, curiosity is evidently confined to them too. An innate reward for curiosity can explain addiction to gambling, which supplies enough uncertainty to be compelling. It can also explain addiction to social networks: refreshing produces new content every time, continually satisfying curiosity.

A Basic World Model

To build a map of the territory it has traveled through, a fish needs two things:

  • An awareness of its own movement.
  • A memory of its relationship to the outside world.

Awareness of movement comes from the vestibular sense. In humans, this involves fluid-filled semicircular canals deep in the inner ear. Neurons monitor the fluid’s movement to determine motion—actually, acceleration. This is why, after spinning in a chair for a while, we still feel as though we are spinning after stopping: those neurons have been overactivated and continue signaling.

Memory of the external world comes from the brain’s medial cortex, also called the hippocampus. Research finds that it activates when a fish swims into a particular spatial region. Visual, vestibular, and head-direction signals reach it, where they are combined and converted into a spatial map.

Breakthrough #3: Simulating and the First Mammals

The Neocortex

Earlier animals had already learned from action, and vertebrates had even acquired TD learning. A new cortex in mammalian brains—the neocortex—would let mammals learn from simulation, or imagination. With this ability, a small mammal such as a mouse could imagine how to get back into a burrow after hunting and avoid being killed, greatly improving its survival chances.

Two conditions are generally thought necessary for this ability to evolve:

  • A broad field of view, allowing the animal to see much of its surroundings and simulate them more accurately.
    • Visibility is much greater on land than underwater. Fish therefore favor reacting quickly to something approaching rather than simulating.
  • Possibly warm-bloodedness: simulation requires much more computation than reinforcement learning in the cortex–basal ganglia system, and neuronal electrical signaling is less efficient at low temperatures.
    • Mammals and birds appear to be the only animals capable of simulation, and they are the only warm-blooded animals.

The neocortex accounts for 76% of the human brain; it is now the brain’s main component. Early research found that it served more than one function, leading to distinctions such as visual, auditory, and olfactory cortex. In the mid-twentieth century, neuroscientist Mountcastle found that different neocortical regions have the same structure and perform the same computation.

Further experiments supported this conclusion:

  • MIT neuroscientists cut off auditory input in snow leopards and rerouted retinal input to the auditory cortex. The animals developed very good vision.
  • Another finding was that the visual cortex of people blind from birth did not become useless, but helped with hearing instead.
    • This explains why blind people are often considered to have especially keen hearing and smell.

Helmholtz’s Theory of Perception and the Helmholtz Machine

How does the neocortex work? The story begins with nineteenth-century German physicist Helmholtz. His theory tried to explain several things the human brain does:

  • Automatically complete incomplete shapes.
  • Perceive only one interpretation at a time, such as seeing an ambiguous image as either a duck or a rabbit.
  • Find it impossible to unsee something once perceived: after recognizing a frog in an indistinct image, we keep seeing it.

Helmholtz argued that we perceive not what we actually experience, but what the brain believes to be present. He called this process inference. We perceive a simulated reality inferred from what exists, not the things themselves directly.

In the 1990s, Geoffrey Hinton and colleagues built a machine to test Helmholtz’s ideas, calling it the Helmholtz machine. It had two parts:

  • Recognition model
    • Infer a latent variable \(z \sim q(z|x)\) from data \(X\). Here \(q(z|x)\) is a prior hypothesis about \(z\) given input \(x\), intended to approach the true posterior \(p(z|x)\) as closely as possible.
    • The recognition model learns \(q(z|x)\).
  • Generative model
    • Use latent variables \(z\) to reconstruct the original \(x\), modeling the joint distribution of \(x\) and \(z\): \(p(x,z) = p(z)p(x|z)\).
    • Here \(p(z)\) is a specified prior, while \(p(x|z)\) is what the generative model learns.

Training uses the wake–sleep algorithm:

  • Wake phase
    • Feed real data \(x\) through the recognition model to obtain \(z \sim q(z|x)\).
    • Hold \(z\) fixed and have the generative model reconstruct \(x\) from it. Compare the result with the original \(x\) to adjust the generative model’s parameters, for example by minimizing MSE or cross-entropy loss.
  • Sleep phase
    • Have the generative model sample \(z \sim p(z)\), then generate \(x’ \sim p(x|z)\).
    • Have the recognition model recognize \(x’\), yielding \(q(z|x’)\), and adjust its parameters toward the true posterior \(p(z|x’)\) by minimizing KL divergence, bringing the distributions closer.

For a concrete example:

  • Wake phase
    • Feed in a picture of a cat, obtain a latent variable through the recognition model, then have the generative model use that information to draw a cat again.
    • The latent information might describe features such as color and body shape.
    • Since the original cat image is available, compare the drawing with it to improve the generative model.
  • Sleep phase
    • Sample from the latent distribution, draw a cat, then have the recognition model recognize this “fake cat” and recover its latent distribution.
    • Since the sampled latent variables are known, compare the two to improve the recognition model.

This resembles today’s VAEs and GANs, and the Helmholtz machine did indeed inspire those modern models.

Helmholtz considered perception largely an inference process: internally simulate the world with a generative model, then compare it with sensory evidence. The success of generative models in machines lends weight to that view, at least demonstrating that similar principles can work.

Some neuroscientists therefore call perception a “controlled hallucination”: the brain continuously generates, while sensory input constrains its generation. A generative model might also explain why we dream. Many other animals sleep, mainly to conserve energy, but only mammals and birds show clear signs of dreaming, with the senses inactive during sleep. In Helmholtz-machine terms, a day spent doing recognition can be balanced by more generation at night. Hallucinations and cognitive impairment after prolonged sleep deprivation might therefore reflect a need to do more generative work.

Research also finds that imagination and recognition cannot operate simultaneously. During imagination, people’s pupils noticeably dilate, and the brain actually stops processing visual data. Recognizing and imagining similar things also activate similar neocortical regions: seeing a cat and imagining one activate similar neurons.

We can therefore think of the neocortex as rendering a model of the environment and continuously making predictions. Anything that disagrees with those predictions produces surprise. Walking on level ground is unsurprising because it matches the model; encountering a hole and finding no ground beneath one’s foot violates the simulation and attracts attention.

Prediction has, in fact, been developing since the bilaterians:

  • Bilaterian reflex circuits predict reflex activation: one neuron’s activity predicts another’s.
  • Vertebrate basal ganglia predict future rewards, as when a monkey uses a geometric image to anticipate sugar water.
  • The mammalian neocortex predicts everything.

Abilities Enabled by the Neocortex

The neocortex’s generative capacity gives mammals several new abilities.

First is vicarious trial and error. Mammals can imagine the consequences of taking a particular course of action. Studies show that rats navigating a maze pause at difficult choices and imagine what would happen along the different routes.

Second is counterfactual learning. One problem in reinforcement learning is that an outcome can be poor but still rewarding. A fish might catch only one small fish, for example, yet receiving food still reinforces the behavior. Machine reinforcement learning applies normalization because good and bad are relative. A mammal can instead imagine how many fish another strategy might have caught and whether that would have been more, reflecting on its behavior. A monkey playing rock–paper–scissors, for instance, tends after a loss to choose the option that would have won the previous round.

Counterfactual learning is a more sophisticated answer to credit assignment:

  • Bilaterians use basic mechanisms such as blocking, latent inhibition, and overshadowing.
  • Vertebrates assign credit across time through the critic’s prediction of changing future rewards.
  • Mammals assign credit through counterfactual imagination: what actually caused the event?

Third is episodic memory. Research suggests that we do not store an entire past scene intact; we simulate it, filling it in during recall much as we complete visual shapes. This is why episodic memories feel real but are not entirely reliable. Simulating the past resembles simulating an imagined future. The neocortex therefore seems central, but the hippocampus also matters greatly. Vertebrates already had a hippocampus, or medial cortex, supporting simple memory, such as a fish remembering nearby space. Its actual role is rapid scene encoding: when new information enters the brain, the hippocampus quickly encodes it as an event pattern, later transferred to the neocortex.

The overall memory process is roughly:

  • Sensory input receives initial processing in the sensory neocortex, forming a temporary simulation of the world.
  • The hippocampus rapidly captures the pattern arriving from the sensory neocortex and encodes an initial memory.
    • This is rough, simple pattern recognition, but very fast: if encoding were slow, the event would already be gone.
  • Over the following hours and days, the hippocampus repeatedly replays the patterns, especially during sleep and rest.
    • Replay strengthens hippocampal synapses, forming relatively stable intermediate-term memories.
  • Later, the hippocampus repeatedly sends the encoded patterns back to the neocortex to establish more stable long-term memories.

Recalling a long-term memory then means simulating it in the neocortex. People with hippocampal damage can therefore lose the ability to form new memories while retaining remote ones, as in the famous case of patient Molaison.

Model-Based Reinforcement Learning

Reinforcement learning alone does not perform well enough in complex board games, so models such as AlphaZero use another technique: searching over strategies. After the opponent moves, they simulate possible continuations thousands of times and select the move with the highest win rate. Rather than choosing one actor’s action directly, this is like letting a group of actors act and selecting the best result—a simple extension of TD learning. We generally call this model-based reinforcement learning (MBRL).

Doing the same in nature is much harder than in Go. In a board game, the search strategy—which actions to simulate—is fixed. Different natural situations demand different search strategies. Human brains flexibly use different strategies, sometimes acting without reflection and sometimes performing extensive imaginative simulation. This is far beyond current AI. How do we do it?

The neocortex is where simulation occurs, while research identifies the prefrontal cortex, particularly the aPFC, as controlling it. Rats with aPFC damage become impulsive and lose planning ability. The aPFC controls simulation by “predicting the self.” The neocortex builds a world model, continuously simulating the world and predicting what external objects will do next. The aPFC builds a self-model, taking inputs from the hippocampus, hypothalamus, and elsewhere to predict what “I” will do next. When its predictions disagree with what happens, it too experiences surprise, triggering simulation. A prediction mismatch indicates uncertainty. Since the aPFC is already predicting several possible outcomes, those outcomes can become the subjects of the next simulations.

Goals, Habits, Working Memory, and Self-Control

In one study, rats learned to lift a lever for food. They were then given contaminated versions of the reward food directly, and most became less eager to lift the lever. This indicates more than simple model-free reinforcement learning: before acting, they simulated the reward and stopped when it no longer seemed attractive. Rats that had previously lifted the lever very many times, however, continued. This is what we call a habit. The basal ganglia have taken over, and the action no longer prompts reflective simulation. A habit is an automated, stimulus-triggered behavior: a model-free case. Habits make many tasks easy, but they become harmful in the wrong context.

A model-free reinforcement-learning system such as the basal ganglia does not really have a goal; it acts to maximize reward. The aPFC—agranular prefrontal cortex—does have a goal, because it plans toward a particular outcome. Shopping in a supermarket, the former system follows the most efficient previously reinforced route. The latter builds a rough layout, simulates possible routes, and chooses among them, requiring an explicit goal such as finishing the shopping sooner. In one sense, a model-free system has a goal during learning too: its reward rules. But those are merely criteria for reward and may not be especially concrete. Asked why an AI took an action, a model-free system can answer only, “Because this action received more reward in the past.” A model-based system can answer, “To achieve this purpose.”

Compare the aPFC with sensory cortex. Sensory cortex has a layer 4 that receives sensory input and initiates simulation. The aPFC has layer 4 during development, allowing external input to establish internal models, but it atrophies as the region matures. Its input no longer consists of raw sensory data, but information from other neocortical areas and the basal ganglia. Integrating these more abstract inputs produces higher-level intentions and goals.

The aPFC also needs working memory: the information required to maintain an intention while carrying it out. It does this by repeatedly reactivating neocortical simulations of the scenes that need to be retained. A rodent searching for nuts among trees, for example, needs to remember which trees it has already visited; its aPFC repeatedly simulates them to avoid collecting from the same ones again. The hippocampus, by contrast, rapidly encodes information for long-term storage and gradual integration into the neocortex. Working memory is temporary and soon forgotten, rather like a computer register. It typically holds information that must be manipulated temporarily and updated in real time: numbers in a math problem, for example, or a sentence’s structure during comprehension. The hippocampus mainly encodes environmental information, contextual details, and an event’s background rapidly.

Another aPFC function is inhibiting the amygdala, which triggers instinctive emotional responses such as fear, desire, and aggression. This inhibition makes human behavior relatively more rational and less instinctive. The aPFC is energetically expensive to operate, so fatigue and stress weaken it, making us more animal-like: impulsive, irritable, and so on.

Breakthrough #4: Mentalizing and the First Primates

Primate Groups

Research finds a positive correlation between brain size and social-group size in primates, but not in other animals. Group living has both benefits and costs. Its main benefit is safety: a herd of zebras can threaten a lion, while a lone zebra is simply prey. The cost is internal conflict over mates and food, consuming precious energy. Social animals therefore generally evolve signaling and submission mechanisms. Deer, for example, lock antlers rather than fighting to the death to determine a leader; the loser submits. Such hierarchies make groups more stable and efficient. Primates are not very different up to this point, but they have a distinctive ability: theory of mind, inferring what others intend and know.

Social rank is extremely important in primate groups, and primates are highly sensitive to its existence and changes. Play a dominant call from a high-ranking member followed by a submissive call from a low-ranking one, and baboons show little response. Reverse the arrangement—a low-ranking dominant call followed by a high-ranking submissive call—and they become alarmed, staring at the speaker to see what has happened. Unlike many other mammals, primates do not base rank simply on strength. It depends heavily on birth, much as in humans. Among females, for example, the highest position goes to the senior female of the highest-ranking family, then her daughter, followed by the senior female of the next family and her daughter, and so on. Changes in family rank depend on the strength of the family as a whole. If a higher-ranking family seems weakened, a lower one may rebel. Both sides may recruit help from outside their own family or group.

This arrangement may be related to a special ecological niche primates occupied after the Permian–Triassic extinction. Living in treetops and obtaining energy from fruit made life relatively easy, with little energy needed for food and ample spare time. Evolving a larger brain for social life was therefore more useful than evolving greater strength. Selection became a social arms race, a contest in political intelligence. The result was a flourishing of many of today’s human social instincts, both apparently positive—friendship, reciprocity, reconciliation, trust, sharing—and negative, such as nepotism and deception. The ability underpinning these new political behaviors was theory of mind.

Modeling Other Minds and Inverse Reinforcement Learning

The earliest mammalian brains weighed less than 0.5 g; the earliest apes’ brains had reached 350 g. What was added? Much of the increase simply enlarged existing regions proportionally, but there were genuinely new parts too. One was the gPFC, or granular prefrontal cortex, surrounding the aPFC. Its novelty lay not in its microcircuitry, which was still the same as other neocortical areas—the gPFC is therefore also neocortex—but in its input/output connections.

The gPFC takes the aPFC’s state as input, constructing an internal model of the self whose function can be described as “explaining intentions.” Recall the layers: the neocortex builds a world model and continuously simulates the world; the aPFC models the neocortex and predicts the self’s behavior to form higher-level intentions for better control. The gPFC goes one step further, predicting the aPFC and the intentions it forms, thereby explaining them.

Put a primate in a maze and ask why it turns left. Different levels of abstraction yield different answers:

  • Reflex: a hard-coded rule directs movement toward the source of a smell, on the left.
  • Vertebrate TD learning: turning left maximizes future reward.
  • Mammalian neocortex and aPFC: it knows food is on the left and intends to get it.
  • Primate gPFC: it knows it is hungry, that eating when hungry feels good, and that food is on the left.
    • Psychologists and philosophers call the ability to think about thinking itself metacognition.

Modeling another mind applies “knowing oneself” to “knowing others”: project oneself into another’s situation to understand them, acquiring theory of mind.

Theory of mind also lets primates learn new skills by observing others’ actions. Many non-primate animals learn by observation too, but only about which choice to make: they discover which of their existing skills to use in a situation. A simple reflex can accomplish that, such as a turtle looking where another turtle looks. Primates can instead learn entirely new skills, things they have never done before.

An important consequence is tool use. Once a primate invents a way to use a tool, witnesses can learn it and pass it down through generations.

In 2010, Andrew Ng and colleagues used inverse reinforcement learning (IRL) to build AI capable of autonomously piloting a remote-controlled helicopter. The approach was to show the AI videos of humans driving, train it to infer the intentions behind their behavior, and let it infer an appropriate reward function. Conventional reinforcement learning could then train it further. What IRL does is, in principle, very similar to what the gPFC does.

Breakthrough #5: Speaking and the First Humans

Human Language Versus Animal Communication

People have argued for human uniqueness since antiquity. Yet human and chimpanzee brains have now been shown to have no structural difference, only a difference in scale. The main reason humans and chimpanzees are so different today may instead be their means of communication: language.

Animals do communicate, but their communication is genetically hard-coded. Other primates raised in entirely different environments can still communicate through calls and gestures when they meet. We can also use words to train and communicate with cats and dogs, but this is basic TD learning: reward a dog during training, and it learns that certain actions after certain sounds predict future reward.

Language learning therefore distinguishes imperative from declarative labels. The language used to train cats and dogs can only be imperative, telling them to do something. Only human language can express declarative labels.

Experiments have also tried teaching chimpanzees human language from infancy. They eventually reached something like a two- or three-year-old human’s level, using simple combinations of words. Some learning is reasonable, since the brain structures are not actually different. The ceiling at a young child’s level may reflect the absence of a learning instinct, discussed below.

What Language Makes Possible: A Singularity

Language’s greatest significance is that it transmits internal brain simulations between individuals, allowing us to learn from other people’s imagination. Compare the earlier ways of learning:

  • Bilaterian reinforcement learning and vertebrate TD learning: learn from actual actions.
  • Mammals with neocortex: learn from imagination, or simulation.
  • Mentalizing primates: learn by observing others’ actions.
  • Humans with language: learn from others’ imagination.

Language also lets humans share imagined worlds and build cohesion: what Harari calls “imagined communities.”

More important is the mechanism itself. DNA’s power lies not in the particular creations it produces—hearts, limbs, and other organs—but in how it operates, allowing natural selection eventually to pick the best combinations. Likewise, language lets humans select and preserve the knowledge most beneficial to survival over generations. Chimpanzees lack language, so useful internal simulations cannot be transmitted and retained. Inventions beyond a certain complexity threshold therefore remain forever out of reach.

Once accumulated knowledge reached a certain amount, a singularity occurred. Humans responded to the growing volume of knowledge in several ways:

  • Evolving larger brains so each individual could hold more knowledge.
  • Distributing knowledge across individuals, with some making spears and others making clothes: specialization.
  • Inventing writing, so that knowledge no longer had to be stored only in brains.

Language in the Brain and a Learning Curriculum

We now know that two brain regions are primarily responsible for human language:

  • Broca’s area produces language. Damage can leave comprehension intact but prevent fluent speech, with severely disordered grammar and only simple meanings expressible.
  • Wernicke’s area supports comprehension. Damage prevents understanding, while speech may remain fluent but meaningless: gibberish.

These areas handle language itself, not a particular form of it. Damage therefore affects speech, writing, sign language, and other forms alike.

Both regions are actually neocortex. We have simply named those locations; structurally they do not differ from the corresponding parts of a chimpanzee’s brain. In that sense, chimpanzees have them too. Why, then, do they not produce language?

We now know that our emotional expressions, rather than language, are the direct evolutionary descendants of chimpanzee gestures and calls. Humans have two systems for controlling the face: an older, emotion-driven reflex system and a neocortical system providing voluntary control. One patient, for example, lost voluntary control of facial muscles but could still smile when happy. People with damage to Broca’s or Wernicke’s area can likewise cry and laugh normally. We have fully inherited chimpanzees’ gestural and vocal abilities as emotional expression; language is something learned afresh.

Language is so complex compared with other skills that it could not be imprinted directly in the genes. Instead, genes impose a complete learning curriculum that leads humans to acquire it. Other animals do similar things. Birds are not born able to fly, but genetically driven wing-flapping and related behaviors gradually teach them. Human infants begin trying to communicate at four months, and at nine months show joint attention, wanting adults to explain what something is so they can label it. Researchers also find that Broca’s and Wernicke’s areas are not intrinsically special: if removed in infancy, language can still be learned, with other neocortical regions taking over their functions.

This may be why chimpanzees cannot truly acquire language: their genes do not encode that curriculum, so they do not keep learning it.

Another difficult question is that language’s early function was altruistic rather than selfish. Our other evolved abilities all strongly benefit the individual’s own survival, while language may primarily teach others. Its evolution would therefore involve group rather than individual selection: groups with language survive better.

Animals already had some altruistic behaviors before language, mainly of two kinds:

  • Kin selection: a bias toward close relatives, readily understandable in genetic terms.
  • Reciprocal altruism: helping when a return is expected.

Language may initially have been used between parents and children, chiefly to teach tool use. It later spread beyond kin to the whole group. People first used it for gossip: sharing others’ moral transgressions and discussing relationship changes. Cheating became more costly and easier to punish; conversely, helping others could receive more reward. Language therefore helped create a system of reciprocal altruism, making higher levels of altruism possible over time. A feedback loop formed: greater altruism encouraged more information sharing; more sharing improved language skills; better language made gossip more effective, further promoting altruism. Our instinct to label people good or bad may also come from this.

The Perfect Storm: How Humans Emerged

The human and chimpanzee lineages diverged about seven million years ago. Brain size remained unchanged until a little over two million years ago, then rapidly tripled over a short period.

Ten million years ago, East Africa was a lush oasis of trees in which our ancestors lived. Tectonic movement later created new mountain ranges extending along present-day Ethiopia: the region we now call the East African Rift. The mountains separated chimpanzees on either side. In the west, conditions remained similar and the animals largely retained their ancestral state. In the east, trees diminished and increasingly open grasslands appeared. Selection began to act, and these eastern apes eventually became humans. Around four million years ago, they still looked much like western chimpanzees, except that they had begun walking upright. Whatever upright walking was for, it did not require additional brainpower; brain size remained essentially unchanged.

By a little over two million years ago, the new grasslands contained many large mammals: elephants, zebras, giraffes, wild pigs, leopards, lions, hyenas, and others. Our ancestors were apex predators there because they had invented tools: Oldowan tools, perhaps flakes of obsidian or quartz produced by striking stone—the word Oldowan means Old Stone Age—and stone axes, among others. These people are called Homo erectus, upright man. The name is rather unfortunate, since they were already walking upright before becoming Homo erectus. They may have hunted through prolonged pursuit, using endurance to catch prey. They evolved features that strengthened that endurance: longer legs, arched feet, hairlessness, more sweat glands, and so on. Cooking was probably also invented then, extracting more nutrition from the same food and providing a larger caloric surplus.

As brains grew, a new problem arose: they could no longer pass through the birth canal. Homo erectus adapted through earlier birth, delivering young before they were fully developed. The brain then needed another twelve years to finish growing. This is why the young of many other animals can stand or even run at birth, while human infants are so much less capable.

Comparing LLMs with Humans

The neocortex reasons by rendering internal simulations, a mechanism that often leads humans astray. Consider a person who likes being alone, enjoys gentle music, and wears glasses. Are they more likely to be a librarian or a construction worker? People usually choose librarian because the description fits the imagined stereotype. But construction workers outnumber librarians by dozens of times, so statistically the latter is more likely. We choose librarian because that image is easy to render, whereas imagining the construction worker is harder. In this kind of reasoning, we fill in characters and scenes while neglecting causal and statistical relationships, a source of many human biases. In a way, this resembles how LLMs work: filling in gaps.

At the level of language alone, however, LLMs merely predict the next word, while humans simulate a world model in the neocortex as they do so. By comparison, the LLM mechanism is much more superficial. Consider the paperclip problem: ask a factory to make as many paperclips as possible, and the request carries many implicit meanings. It does not mean turning the entire planet into a paperclip factory, for example; the actual purpose may be making more money, not acquiring paperclips for their own sake. Such requests carry subtle constraints. Understanding them requires, above all, modeling the requester’s mind: the intelligence acquired through mentalizing.