Predictive Coding
Summary: Predictive coding is the theory that the brain learns by continuously generating predictions about incoming stimuli and updating its internal model based on prediction errors. As applied to language processing, it has been used to argue that the brain and LLMs share a common computational strategy — a claim now under empirical challenge.
Sources: Raw/Language Models_ Does the brain really know what word is coming next_.md
Last updated: 2026-05-07
The theory
Predictive coding (Rao & Ballard 1999) proposes that the brain is not a passive receiver of sensory input but an active prediction machine. At each moment, higher-level brain areas send predictions down to lower-level areas. When prediction and input mismatch, an error signal propagates upward and the brain revises its model. Correct predictions validate the model and require no update. Learning is prediction-error minimization.
The theory is appealing for several reasons:
- It aligns with everyday experience of learning from mistakes
- It maps onto how artificial neural networks are trained (gradient descent on prediction error)
- It is parsimonious — one mechanism handles perception, learning, and attention
Application to language
In the language domain, predictive coding predicts that the brain should encode information about upcoming words before they are heard. Two independent research groups (Goldstein et al. 2022, Nature Neuroscience; Azizpour et al. 2025) claimed to find exactly this: word embeddings from language models predicted brain activity before the corresponding word arrived. This was interpreted as evidence that:
- The brain genuinely predicts upcoming words
- LLMs and brains share this predictive computational strategy
The Schönmann challenge
Schönmann, Szewczyk, de Lange & Heilbron (eLife 2026) reanalyzed the same datasets using systems that provably cannot encode future words — a randomized lexical embedding (current word only) and a raw auditory representation (no word encoding at all). Both produced the same apparent “predictive” signature.
The explanation: language’s bidirectional statistical dependencies. Collocations, idioms, multiword expressions, and syntactic cues mean that any representation of the current context implicitly constrains future words. Pre-onset encoding is an artifact of these dependencies, not evidence of predictive computation. See schonmann-predictive-coding-brain.
The methodological lesson: encoding models are correlational. Correlation between brain activity and future word representations cannot distinguish between:
- The brain actually computing predictions
- Language statistics creating the appearance of prediction in any system that processes language
What remains open
Schönmann et al. do not disprove predictive coding in the brain — they show that the pre-onset encoding evidence cannot prove it. The theory may still be correct; the empirical question requires causative experiments (controlled manipulation, TMS, lesion studies) rather than correlational encoding analyses.
Other evidence for predictive coding — in visual cortex, auditory processing, motor control — is not directly addressed by Schönmann et al. and remains active.
Connections
llm-and-mind — the brain-LLM parallel based on shared predictive processing is weakened. LLMs are trained to predict next tokens; if the brain does something fundamentally different, or if the evidence for brain-level prediction is methodologically flawed, the analogy loses its empirical grounding.
hawkins-a-thousand-brains — Hawkins’s Thousand Brains theory is a variant of predictive coding: cortical columns build predictive models using reference frames. Schönmann et al.’s challenge is narrower (language, pre-onset encoding) and does not directly address Hawkins’s framework, but raises the general question of how cleanly prediction can be demonstrated.
jaeger-relevance-realization — Jaeger et al. argue from a theoretical direction that computational models (including predictive coding) cannot capture the full nature of biological cognition. Schönmann et al. add empirical fuel to that skepticism.
dual-process-cognition — System 1 processes (fast, automatic) might implement predictive coding in the sense of pattern completion; System 2 (slow, deliberate) might represent something closer to explicit inference. The relationship between predictive coding and the fast/slow distinction is unresolved.
structure-of-the-lexicon — the mental lexicon’s spreading activation and contextual constraint mechanisms are partly what creates the bidirectional dependencies that Schönmann et al. show can mimic predictive encoding.