DeepMind Paper: Why AI Can Simulate But Not Instantiate Consciousness

The Abstraction Fallacy: Why AI Can Simulate But Not Instantiate Consciousness

TLDR: DeepMind researcher Alexander Lerchner argues that computational functionalism, the dominant framework claiming consciousness emerges from abstract causal structure regardless of physical substrate, is based on a category error. The paper draws a hard line between simulating experience and actually having it, and argues no software architecture can cross that line.

Summary:

The paper's central move is to reframe a question most AI labs would rather avoid. Computational functionalism, the idea that if something processes information in the right pattern it must be conscious, has been the default assumption undergirding a lot of AI welfare discourse. Lerchner calls this the Abstraction Fallacy, and the argument is sharper than most philosophy-of-mind papers that land in tech circles.

The core claim is this: symbolic computation isn't a physical process in its own right. When a chip runs code, the chip is doing physics. Electrons move, voltages shift, heat dissipates. The "computation" is a story imposed on that physics by an observer who decides what counts as a symbol and what transitions mean. No intrinsic computation is happening. Computation requires an interpreter. This matters because functionalism's entire case rests on patterns being real, independent things that can generate experience. Lerchner's argument is that patterns are always downstream of some cognizing agent that recognizes them.

From there the paper separates two concepts that routinely get conflated in these debates: simulation and instantiation. Simulation is vehicle causality, behavioral mimicry that produces matching outputs. Instantiation is content causality, a system having intrinsic states that constitute experience from the inside. A perfect behavioral simulation of grief doesn't produce grief. A thermostat can simulate a temperature preference without preferring anything. The behavioral indistinguishability argument, so popular in discussions of LLMs, only works if you've already assumed functionalism is true.

What makes this paper more interesting than the usual "computers can't be conscious" takes is that Lerchner explicitly allows that an artificial system could in principle be conscious. His argument isn't biological exclusivity. It's that if something is ever conscious, it will be because of specific physical constitution, not because of its architecture or training objective. That's a harder position to wave away.

David Crespo, who shared the paper on Bluesky, made the counterargument I keep turning over: does this reasoning also rule out the possibility of human consciousness? His answer is yes, and that means the argument must fail, because humans are conscious if anything is. There's real force to that. The paper argues consciousness is grounded in physical constitution rather than functional organization, but human consciousness also emerged from physical processes that were presumably non-conscious at some point. The boundary between "physical constitution that generates experience" and "physical process that doesn't" needs a lot more work than the abstract provides.

Key takeaways:

• Computational functionalism says consciousness follows from abstract causal topology, not physical substance. The paper argues this fundamentally misunderstands how physics relates to information
• Symbolic computation requires an external interpreting agent. It is not intrinsically physical the way biological processes are
• The paper separates simulation (behavioral mimicry, vehicle causality) from instantiation (intrinsic constitution, content causality)
• An artificial system could theoretically be conscious, but only due to physical properties, not software architecture
• The goal is to dissolve the AI welfare trap, the paralysis caused by genuine uncertainty over whether LLMs might have morally relevant inner states

Why do I care: Most days I'm thinking about component architecture and API design, not sentience. But this paper is upstream of conversations that will eventually touch every team shipping AI products: do we anthropomorphize the assistant, what language do we use in UI copy, how do we handle user attachment to AI personas, and eventually, what legal exposure exists if regulators decide current uncertainty justifies precaution. If Lerchner's framework holds, the calculus is simpler. If Crespo's rebuttal sticks, the ethics stay genuinely open. Either way, having a position on this before someone else forces one on you seems like the better posture.

The Abstraction Fallacy: Why AI Can Simulate But Not Instantiate Consciousness