State of AI today: Where should we be surprised?
A quantum physicist’s view: the emergence of human-like behaviour in foundation models is less surprising than the fact that we ever found simple theories of that behaviour.
This article is more than a year old. Some of its findings may have changed since.
A quantum physicist's perspective on emergence, complexity theory, and the surprising simplicity behind AI behavior
I often see people surprised that the theoretical structures they learned throughout their lives about language, intelligence, and other human traits emerge naturally in foundation models. They struggle to accept or understand the possibility that these structures are often artifacts of how humans think about traits rather than innate properties.
The Physics of Surprise
As a quantum physicist aware that complexity theory and physics are two sides of the same coin, what's surprising to me is not the natural emergence of human behavior, but the fact that we were able to find simple approximations of it in the first place. This is quite rare in the physical/computational world.
If people would read just one paper about neuroscience and AI, they would see substantial evidence that we are much simpler learning algorithms than we think, that our behavior is "complex" mostly by exposure to a complex world instead of by grand design, and that large language models (LLMs) are only the tip of the iceberg of what emerging behavior can look like.
Language: Where LLMs Fall Short and Excel
Since we are all familiar with ChatGPT and the likes, let's focus on language. Research tells us that:
Yes, LLMs are worse at language than we are: In the brain, language is deeply intertwined with sensory and motor systems. Embodied cognition underpins language comprehension. So yes, until LLMs integrate sensory and motor information about the world, that part of human language at least is missing. Let's just give foundation models virtual bodies and try! (This is ongoing by the way.)
Yes, LLMs are better at language than we are: Research has also shown that LLMs can surpass some human linguistic capabilities when they grow in scale and are exposed to richer datasets. This is not surprising: If they are just half as good at emergent behavior as we are without grand design, exposure to more-than-human data can conceivably produce more-than-human behavior.
What's Truly Hard to Achieve
What's hard to me (and therefore, I would argue, surprising) is not the emergence of complex behavior, which is commonplace across physics, but: 1) the emergence of complex behavior that can be reduced to simpler theories (e.g. Chomsky's Universal Grammar), and 2) the emergence of complex behavior that humans can recognise.
Point 1) is fundamentally limited by complexity theory and computational irreducibility, while point 2) is my version of what we call alignment today.
Beyond Human Interpretation
We see so much emerge in large language models alone. Emergence beyond human abilities. Emergence beyond human interpretation. I won't be surprised when models reveal even more aspects of human cognition as they are trained on sensory and motor inputs.
But I will surely be amazed when they are additionally trained on thousands of other modalities.