Don't Let ChatGPT Set All Agendas
The MIT study “Your Brain on ChatGPT” tests LLMs inside interfaces built for token consumption, not for cognitive engagement. The essay calls for real-world observation and for not letting one platform shape research, design and policy on AI.
This article is more than a year old. Some of its findings may have changed since.
From study protocols to geopolitics: why one platform shouldn't define our AI future
Studies like the MIT Media Lab's "Your Brain on ChatGPT" ask important questions but provide few actionable insights.
The Problem with Brain Measurements
While EEG can reflect task-related modulations in connectivity patterns—often interpreted as markers of cognitive engagement—these patterns should be interpreted cautiously. They do not directly measure executive function per se, but may index neurophysiological correlates or modulators of it.
Artificial Studies vs Real-World Use
Regardless of brain measurements, the study protocol bears little resemblance to sensible real-world uses of ChatGPT—let alone the broader spectrum of LLM applications. People should spend less time confirming their own biases with artificial studies and more time designing and documenting real-world experiences.
Daily observational studies are worth far more than a few randomized control trials with artificial constraints.
What These Studies Actually Probe
Beyond the many study misinterpretations (which many others have pointed out), one point is clear: such studies don't probe LLMs optimized for cognitive engagement—they probe LLMs integrated into poorly designed interfaces optimized for token consumption.
It's straightforward to frame LLMs as partners or "co-thinkers," controlling the balance between "you think" and "I think." As AI Swiss has observed repeatedly in real-world settings, the sweet spot often lies in the middle: where the LLM listens, challenges, helps identify dead ends, enlarges horizons when you're stuck.
Stop Letting ChatGPT Define AI
We need to stop letting ChatGPT and similar platforms dictate our perceptions about LLMs (or even AI!), our study protocols, our UX/UI designs, our investments, and even our regulations and geopolitics.
AI Swiss' white paper attempts to set a more fruitful direction for societal thinking: exploring human-conversational-AI interactions as a new science of communication, cognition, and development, to teach new transversal skills as early as possible in life, independently of digital or data literacy, or even conventional AI literacy.
Talk to Rodin if you want to know more.