Hallucinations, Interpretability, and the Seahorse Mirage

A follow-up in the past-tense series testing whether U-shaped learning, model size, or measurement explains irregular verb behavior—and finding the language matters too.

This episode challenges the old idea of a single language box in the brain, using aphasia studies and hospital control groups to show how language processing is more distributed than once thought. It also explores how different languages rely on word order or verb marking in different ways, and why those differences matter when the brain is under stress.

Thirty years ago, the authors of Rethinking Innateness argued that grammar could come out of a learner with no grammar built in. Nobody could check it. Now there's an instrument. A reading of the Inside the Black Box essay on what Elman, Bates, and Karmiloff-Smith would make of transformer language models — attention as a lookup, structure nobody installed, and the part that isn't there.

Can a model learn past tense the way children seem to, if you remove the tokenizer from the equation? This final installment tests raw letters and raw sound, probes whether a U-shaped pattern can be forced, and shows why the answer points to distributed, frequency-driven learning rather than a discrete rule.

Part 3 of 4. After two episodes arguing there is no discrete past-tense rule inside a language model, this one turns the strongest possible search on the question: gradient descent, hunting for the single internal direction that best forces goed over went.