Present-day LLMs, such as ChatGPT and Claude, can perform complex tasks, such as writing poetry and solving difficult algebra problems, with astounding speed and precision. Some researchers have referred to this phenomenon of AI acquiring startlingly human-like skills as "emergence." But not everyone agrees with this terminology.
Emergence occurs in nature when individual units in a complex system interact to produce unique behaviors qualitatively different from those of the units themselves. For example, birds seamlessly create unique formations to prevent collisions as they fly together across the sky, and communication across billions of neurons in the brain gives rise to sophisticated capabilities, such as language and memory. In a recent paper in a themed issue of Philosophical Transactions of the Royal Society A, SFI researchers have put claims about emergence in AI to the test. "Just getting better at something doesn't make you have an emergent property," says External Professor John Krakauer (The Johns Hopkins University School of Medicine), who co-authored the paper along with SFI Professor Melanie Mitchell and SFI President David Krakauer.
The authors propose a framework grounded in complexity science to clarify whether a system exhibits emergent capabilities. For example, one of their criteria concerns the mechanisms of scaling. As an LLM grows in size, its performance inevitably improves. But that's not sufficient for emergence to occur. "It needs to show a qualitative change in the internal mechanism with respect to how the task is done," says John Krakauer. For instance, is the LLM improving simply because it's gorging on more data, or is it actually learning to connect the dots across several ideas and simplifying that association by describing a novel concept? Similarly, proponents of AI claim that certain AI capabilities are emergent because they have appeared suddenly. The paper's authors point out that these supposedly unexpected skills aren't accompanied by any identified internal reorganization of the LLMs.
Moreover, the authors have emphasized the difference between emergent capabilities, which LLMs can arguably develop, and emergent intelligence, which, to date, has not been achieved by AI systems "A collection of capabilities is not intelligence," says Mitchell. "We think of intelligence as a more general ability to take a concept and adapt it to new situations that you haven't been trained for and to acquire some new ability very efficiently, which these systems in general don't do."
Whether AI shows emergence is part of a larger debate around how LLMs function. For example, human intelligence is hypothesized to rely on world models — internal programs of cause and effect in different settings — that help them achieve their goals, avoid harm, and basically survive in their environment. But the concept itself is contested. "It is not at all clear to me what a world model is — a term far more popular than it is useful," says John Krakauer. In the themed issue, “World models in natural and artificial intelligence,” researchers wonder whether current AI models have developed world models and are drawing on them to solve problems given to them by humans. "Perhaps the best way to think about a world model is as an emergent explanation, in which, “less is more” says David Krakauer.
Unfortunately, we don't fully understand how humans develop these world models either. A different paper in the themed issue co-authored by External Professor Alison Gopnik (University of California, Berkeley) seeks to demystify the process. Gopnik and her colleagues conducted two experiments in adults and children. They showed that a process called “empowerment gain” facilitates how humans learn to associate their actions with consequences. Simply put, we sharpen our understanding of the world by engaging in activities that allow us to control and vary their outcomes to some degree.
Mitchell says that understanding the kind of world model an AI has — if it has one at all — would help us better understand why it sometimes shows erratic behaviors such as hallucinations and may even enable us develop better AI in the future: "If an LLM has a human-like model of the world with all of the understanding that goes with it, it might help it become more trustworthy and also explainable to people."