Join

All Things Emergent

A Reality Check on Current AI "Breakthroughs"

Few fundamental realisations have shaped human-like AI: capabilities emerge from exposure to text, then to problem-solving tasks. Most announced breakthroughs, such as DeepSeek R1 or s1-32B, are engineering refinements.

· By Charles-Edouard Bardyn · 2 min read

This article is more than a year old. Some of its findings may have changed since.

Few fundamental realizations have shaped human-like AI. Everything else is engineering refinements.

Amid the relentless flood of overhyped AI "breakthroughs," it's time for a collective reflection on what matters in AI progress. The recent excitement around models like DeepSeek's R1 or Stanford's s1-32B, "reasoning" from reinforcement learning or a few training examples alone—show the limits of our current discourse around AI.

Let's be clear: Such works are important. They show that smart engineering tricks can go a long way in bootstrapping model capabilities. But treating isolated demos like deep paradigm shifts misses the forest for the trees.

The Fundamental Realizations

In reality, few fundamental realizations have shaped the trajectory of human-like AI (a tiny part of what happens in AI):

  1. When you expose learning algorithms to vast amounts of text, capacities that resemble language understanding and world models naturally emerge. And the emergence is much richer when models are further exposed to images, videos, or physics simulations.
  2. When the same models are exposed to problem-solving tasks like mathematical problems, capabilities that resemble algorithmic thinking or reasoning additionally emerge.
  3. That's it. And concepts of universality from computer science and physics make such emergence mostly independent of the exact learning architecture we use for that.

Everything else—including the weekly parade of papers about frameworks architectural tweaks—is engineering refinements to reduce cost and increase performance which is, with markets, what drives AI progress today.

The Physics and Neuroscience Perspective

This shouldn't surprise physicists or neuroscientists. The brain itself is a pattern recognition machine that develops increasingly sophisticated capabilities through exposure to data. The difference is that brains benefit from exposure to a much richer world, with an architecture that's likely not the best but what Nature optimized.

So while it's exciting when "David" (s1-32B and the likes) outperforms "Goliath" (OpenAI o1 and the likes) on specific benchmarks, we must maintain perspective. Cool new tricks do not replace the hard work that goes into developing usable models on a wide spectrum of human problems, or established techniques from physics like MCTS. Training on math problems is much easier than training on non-axiomatic human tasks that depend on a deep understanding of our world. And when it comes to leveraging human world views while enabling decent user experience and risk management, Goliaths like OpenAI are alone.

What Real Breakthroughs Would Look Like

Real breakthroughs would be concrete actionable plans to educate people on scientific reasoning and co-reasoning with AI. Plans to help all the companies that struggle to create value from the massive capex in AI infrastructure. Plans to keep environmental promises despite AI growth. When will the media focus on what matters, instead of relaying talking points?