Analyzing OpenAI o1 "Reasoning": Beyond the Hype
To analyse OpenAI o1 “reasoning”, the essay distinguishes reasoning that emerges from exposure to the world from reasoning added by human-crafted algorithms, and argues that o1 is mostly the second kind.
This article is more than a year old. Some of its findings may have changed since.
A critical examination of cognitive abilities in AI systems and the distinction between emergent and engineered reasoning
You want to analyze OpenAI o1 "reasoning" and other "cognitive" abilities? A word of advice: you won't get very far playing with the model, and you won't go very far drawing parallels to human behavior either.
The Problem with Defining "Reasoning"
First, there are as many definitions of "reasoning" as there are human beings. Many people have attempted to uniquely define such concepts, and everyone still has a different notion in their brain—unique to their brain dynamics. The same goes for "intelligence", "consciousness", and other language artifacts from millennia of anthropocentrism.
Two Types of Reasoning
Second, even if we reduce "reasoning" to a clearer subset of behaviors, we still have to distinguish at least two types of reasoning:
- Reasoning that emerges from exposure to the world (in the case of LLMs, to almost all text of the world)
- Reasoning that emerges from the implementation of human-created algorithms like chain of thought, reflection, evaluation by another model to guide reinforcement learning, Monte Carlo search, etc.
As current progress in research and industry is almost exclusively due to 2), it is almost certain that o1 is like that. This is likely not an algorithmic revolution, but a user experience / engineering one.
The Reality Behind o1's Performance
The jump in performance that people experience with o1 is them catching up with the reality of what human-crafted algorithms bring to the world of AI. In research and industry, LLMs are almost never used without human-crafted algorithms on top. This is how people build all the incredible applications we see today. With o1, such algorithms have likely become part of the new standard experience on simple UIs like ChatGPT. And yes, since this relies on calling multiple LLMs multiple times internally, you have to wait a bit more. For now.
Human vs. Artificial Reasoning
Now, do human brains rely on 2)? Likely not, and almost certainly not like our current algorithms. It would be a very long stretch to imagine Monte Carlo search hardcoded in the brain! Yet similar abilities may very well emerge from exposure to our actual world, i.e., from exposure to a world where one receives inputs from senses, in which one can move and interact, and where language is not the world, but only a representation of it.
No one can say what will happen when models are exposed to such world. Perhaps that 2) won't be needed anymore. More likely, it will help us go much beyond human abilities.
The Impressive Reality
It is impressive that we can get so far in computational abilities (for lack of a better word) with hybrid LLM / human-crafted algorithms. It is even more impressive that humans and animals reach good performance on similar tasks by pure emergence from exposure to the world.
A Note to Journalists
To all journalists out there with the hard task of understanding this complex field, please do not use social media, including this post, as one of your sources. There is a lot of good material out there. You have no excuse. Here is a start.