Exorcising the Ghost in the Machine

Critiquing AI Consciousness - a conversation with Prof. Anil Seth

This essay and interview is part of a wider series critiquing the institutions and ideologies underlying AI development. Special thanks to Anil Seth for the interview.

Can AI ever become conscious? Several researchers have launched an “AI welfare” research paradigm, arguing that LLMs may soon become conscious, sentient, and deserving of protection from potential harms. While sympathetic to such researchers’ precautionary impulse, I believe that their probability calculations rest on contested metaphysical assumptions which their methodology does not adequately interrogate.

This essay describes key features and weaknesses of the research paradigm, before presenting the objections of one of the strongest interlocuters within consciousness science.

The AI Welfare Ecosystem

Researchers investigating AI consciousness and welfare comprise a tightly-connected set of institutions, each of which broadly share assumptions and methodologies.

Eleos is arguably the leading institution, directed by Robert Long, an affiliate of New York University, who was previously employed by the Future of Humanity Institute and the Center for AI Safety.

Anthropic’s Model Welfare Program has been led by Kyle Fish - formerly at Eleos - since 2025. Eleos and Anthropic share advisory staff, and regularly collaborate in evaluating the welfare of models.

The NYU Center for Mind, Ethics, and Policy is directed by Jeff Sebo, and NYU and Eleos staff have authored the field’s flagship report.

Jonathan Birch, a professor at LSE, is one of the field’s academic leaders. The Edge of Sentience, 2024, presented a strong case for policy-makers to take a precautionary outlook in favour of sentience when we consider edge cases and high uncertainty. Birch’s own thoughtful manifesto distinguishes between the dual challenges of misattributing consciousness to LLMs and failing to recognise consciousness when it emerges.

This ecosystem can trace its intellectual legacy to the Effective Altruist movement and its animal welfare communities. This is reflected both in terms of its justifications - research pathways are defended based on notions of neglectedness, expected value, and longtermist assumptions - and in its primary funding sources. The field was largely sourced by Coefficient Giving (previously called Open Philanthropy) in 2021, before receiving additional grants from Anthropic in 2025 and Jaan Tallinn’s Survival and Flourishing Fund in 2026. Academic workshops that connected the field’s pioneers were funded in 2023 by Open Philanthropy, Effective Ventures, and the EA Long-Term Future Fund (Butlin et al: 2023, front matter). The presence of a dense network of funding, hiring, and co-authorship relations between these few institutions has been noted by other commentators.

The field’s core manifesto - ‘Taking AI Welfare Seriously,’ was published in 2024 by Long (Eleos), Sebo (NYU), Birch (LSE), Fish (Anthropic), and other scholars predominantly sourced from Eleos and NYU. They wish that AI developers take seriously the proposition that AI systems might be moral patients and welfare subjects, and acknowledge that there are risks of over-attributing and under-attributing welfare to such systems (see also). If AI systems will soon become conscious and sentient, then their mistreatment and even death becomes a catastrophic moral hazard.

The ecosystem’s small and tightly-connected nature comes with potential concerns. With perhaps fewer than sixty full-time researchers, the paradigm trains, hires, and networks a small set of scholars with similar institutional genealogies and worldviews. This means that Caviola and Saad's expert surveys are only as informative as the diversity of the consulted experts: while 90% of researchers in the paradigm purportedly believe that AI consciousness is possible, yet this uniformity reflects shared priors, theories of consciousness, and institutional affiliations. The dominant methodologies of the field reflect a particular epistemic culture stemming from effective altruism: welfare consequentialism, computational functionalism, and expected value calculations. While this is not necessarily detrimental to the field’s validity, it produces conditions in which independent validation is structurally difficult.

The Selectivity of Butlin et al (2023).

The central paper grounding the ecosystem’s methodology is Butlin, Long, et al.’s (2023) ‘Consciousness in Artificial Intelligence: Insights from the Science of Consciousness.’

The paper adopts what they term a “theory-heavy approach,” in which indicators of consciousness are derived from several scientific theories of consciousness (TOC), (pp. 13, 46). Their confidence in whether an AI system is conscious depends on the similarity of the system’s computational processes to those posited by one of their preferred TOCs, their confidence in that TOC, and their confidence in “computational functionalism” - the broader set of theories that contends that consciousness can be multiple realised by computational processes, (p. 17). They focus on three computational functionalist TOCs: Recurrent Processing Theory, Global Workspace Theory, and Higher-Order Theories. It is from these theories that their indicators of consciousness for AI systems are drawn. The indicator framework is constructed in computational terms, and so will tend to find that computational systems will satisfy it - and so conclude that such systems are likely conscious.

Their methodology and results are highly dependent on computational functionalism, and they reference scepticism to their category of preferred theories only in passing, (p. 70). Commonly debated theories, such as Integrated Information Theory, are dismissed alongside biology-dependent theories. To illustrate just how selective their preferred sampling of TOCs is, it is worth considering the broader map of TOCs, as supplied by Robert Lawrence Kuhn (2024):

Various TOCs are unfriendly to the view that AI systems - and LLMs in particular - could be conscious: Anil Seth’s biological naturalism, in which consciousness is a feature of biological systems (Kuhn, p. 60); George Lakoff’s embodied cognition view, Andy Clark’s active externalism, and Alva Noe’s out-of-our-heads externalism (Kuhn, p. 68); as well as theories that require quantum processes (Kuhn, pp. 90, 151). Wider metaphysical positions in philosophy of mind, such as substance dualism - which contends that consciousness requires immaterial aspects, Keith Frankish’s illusionism - which denies the existence of phenomenal consciousness, David Bentley Hart’s idealistic monism (Kuhn, p. 109; Hart: 2025), and Philip Goff’s constitutive panpsychism (Kuhn, p. 102), are not considered. By excluding each of these positions ex ante, Butlin et al only consider indicators of consciousness which AI systems would theoretically satisfy, and ignore contradicting results.

Various TOCs under the umbrella of biological naturalism argue that consciousness depends in some way on biology. As Ned Block describes, even if you regard consciousness in computational terms, consciousness may require sub-computational biological realisers, such as electrochemical processing. As we shall see, Anil Seth argues that consciousness is connected to the body’s continuous modelling of itself as a living system, and is caught up on interoceptive inference, allostasis, and metabolic self-regulation. Evan Thompson’s autopoietic enactivism would hold that LLMs could not be conscious as they do not maintain themselves against thermodynamic degradation. Similarly, Karl Friston’s view requires genuine homeostatic self-regulation and existential stakes. Antonio Damasio’s “somatic market hypothesis” contends that consciousness is grounded in the brain’s representation of the body’s internal states. Accounts in evolutionary psychology maintain that phenomenal consciousness is a specific biological adaptation that makes experiences feel thick, such that organisms care about being alive - LLMs lack the purportedly necessary phylogenetic distribution and evolutionary pressures.

Another evolving category is quantum theories of consciousness, which hold that consciousness depends on specific physical processes that are either non-computable (Penrose’s Orch-OR), analogue (McFadden), or else are tied to waveform collapse dynamics (Stapp). Transformer architectures are classical systems, and so consciousness tends to be precluded under many quantum TOCs. For instance, McFadden’s Conscious Electromagnetic Information (CEMI) theory claims that an electromagnetic field generated by neuronal firing integrates information through wave interference, and is instrumental for conscious processing. Current LLMs compute through matter, not electromagnetic field dynamics, and so lack the requisite architecture to instantiate consciousness under this view. Such theories remain highly contentious, yet given quantum biology’s recent successes in explaining other biological processes such as magnetoreception and photosynthesis, they should not be excluded without comment.

The TOCs which Butlin et al use - Higher-Order Theories, Global Workspace Theory, and Recurrent Processing Theory - are themselves controversial (Seth and Bayne: 2022). Given empirical immaturity, it is perhaps over-ambitious to derive probability estimates and expected-value calculations from them.

A Conversation with Prof. Anil Seth.

Anil Seth has emerged as one of the most vocal and sophisticated critics of claims concerning AI consciousness. As a Professor of Cognitive and Computational Neuroscience at the University of Sussex and Director of the Sussex Centre for Consciousness Science, he has led research in predictive processing accounts of perception.

In a 2025 paper and a 2026 essay, he has argued that computational functionalism forwards a specious metaphor, “brain-as-computer,” an oversimplification that neglects the complexity of neurological processes. Consciousness is to be understood in light of several features of living systems: living systems are autopoietic, continually regenerating their own material components and actively maintaining boundaries between self and environment; interoceptive inference and allostasis involves living systems regulating their own metabolism, requiring intricate predictive processing to regulate internal conditions.

Given the complexity of Seth’s biological naturalism, I sought to ask him several questions as to clarify his arguments, his evaluation of the Butlin et al methodology, and his intuitions concerning broader debates in the governance of AI.

BeneathTheChip: In your recent TED Talk (Seth: 2026), you remark that “I think we can draw a direct line from the molecular furnaces of metabolism … all the way to the neural circuits that underlie each and every experience we have.’ What biological processes do you regard as necessary conditions for consciousness?

Anil Seth: I think that’s an open question. For me, it’s important to separate the debate into two buckets. The plausibility of conscious AI depends on this assumption called computational functionalism. That’s a very strong assumption, that substrate-independent computations are sufficient, and it’s enough for that to be wrong without biological naturalism having to be right. If biological naturalism is right, then computational functionalism is clear wrong.

One place to start is asking what the most basic kind of phenomenology is. I don’t think its language of complex cognition. That’s almost certainly not how consciousness evolved. It’s not necessarily its most evident functional utility.

[Another place] is this through-line from predictive processing to the free energy principle in metabolism. I’m intrigued by these connections that you have between the minimisation of thermodynamic free energy during metabolism - a really fully-embodied process, and then this more abstract of minimisation of prediction error in perception. There’s a lot of mathematical, substantial evidence that it’s the same thing described in two different ways. … From there we can derive this fundamental process of autopoiesis and metabolism and thing that keep persisting and maintain and regenerate their own components and their own boundaries.

BTC: Can predictive processing not be described using Bayesian equations?

A: You’re right to say that you can abstract aspects of its functional organisation and run it in silico, yet there is no guarantee that this will re-instantiate everything associated with that process as it’s unfolding in biology. … The fact that you can abstract computation doesn’t mean that it’s implemented in an algorithmic way in the brain. In fact, it almost certainly isn’t.

BTC: Since the release of Butlin and Long’s 2023 paper, have these writers interacted much with critics and biological naturalists?

A: Yeah, sure, I mean, I‘ve spoken a number of times to Robert Long and Jeff Sebo and Patrick and so on. I think one of the interesting things at the moment is there is, there is a dialog, and there is a wide range of opinion. [A few years ago] often there was just this really strong assumption that, well, of course, “computational functionalism has to be true” among people who were inclined that way anyway. “Well, what else can it be?” Things have definitely moved on and now, even in the Butlin and Long first preprint about AI and consciousness, they said, “Look, we have to assume computational functionalism instrumentally. Otherwise we can’t do anything.” Every claim is conditioned on this assumption of computational functionalism. They don't interrogate it further, but at least they named it and said it's an assumption, which was major progress.

We should provide a more positive reason for why that should be the case. And the same goes for biological naturalism too. … And I've seen this with this BBS target paper that I published last year, and I'm still writing the response for it. There are like 250 submitted commentaries, and 50 they're publishing, and there all kind of middle grounds that are being explored.

BTC: Jonathan Birch, in his centrist manifesto, is trying to propose ways in which we might falsify biological naturalist theories, such as by seeing whether the proposed necessary conditions are present in species which we have some reason to believe are conscious. Are there any particular empirical tests which you’d like to see performed in the next few years?

A: There are all kinds of empirical questions that one might investigate. If you merely look for how substrate dependent cognitive and functional properties of the brain are, that immediately speaks to whether computational functionalism is likely to be true, because the more substrate dependencies you ding, the less plausible it is that the relevant level of description of the brain adequate to capture all those properties is computational, which undermines computational functionalism. So for instance, there's work we're doing in collaboration with Mike Levin and Tufts, looking at the functional dynamical properties of biological material. Others are doing great stuff on metabolism, mitochondria, and anaesthetics. Electromagnetic fields, I think, are another interesting area that has been a bit overlooked, where there’s a strong substrate dependency.

BTC: You have also said that once we continue to refine our understanding of the brain, this will also help us in terms of making the hard problem of consciousness less mysterious in some ways. Could you elaborate on that?

A: It may be the case that as we are able to explain experiential properties in terms of mechanisms in ways that go beyond brute correlations, then the apparent conceptual mystery, metaphysical mystery posed by the hard problem may seem less severe, and the historical parallel I always use for that is life. I don't think the two questions are the same, so I'm not relying on it. But just as a matter of historical parallel, there was a kind of sense of dualistic mystery about life, which motivated the philosophy of vitalism. And that's gone away. And it's gone away partly because of this refocusing of the effort on not looking for a spark of life, or trying to have a theory of what the élan vital is, but just, okay, look, let's explain properties of living systems. And, okay, there's no hard problem of life.

BTC: Suppose, looking forward, that the debate on AI consciousness continues for the next decade or so, what would you see as some ideal outcomes, whether that be in terms of further dialogue or more empirical tests done? How would you like to see the debate progress in this space?

A: I think so the debate could do with a bit more, I think, nuance and less dogmatism on any particular side. I certainly think that debates about conscious AI should not simply assume computational functionalism anymore. I mean, they really need to be a little more humble about about that and but also argue for it independently, rather than just “what else could it be?” And it may be that there's some interesting middle ground, which is the sort of thing that philosophers like Gualtiero Piccinini suggest, which is that, okay, some of the things that brains do can be described as inherently computational in a way that is substrate independent. So for instance, some of what my brain is doing in perceptual inference may be inherently computational in a way that can be realized in Bayesian machine learning, but other aspects of it may not be. And so we have to figure out which, which is which, and which is associated with consciousness and which isn’t.

I think it's absolutely key there's a whole positive research agenda [concerning biological naturalism] ranging from experiments of the sort we discussed about substrate dependency to more theoretical work trying to really tie down what the mathematical through line is, from things like autopoiesis and metabolism to predictive perception and so on. And alongside that, I think there's a lot of empirical work about human tendencies to attribute consciousness. … Also more work on hybrid computational systems, organoids and things like that. Do we see signatures of consciousness in neuronal organoids? … One thing I interested in is one of the key differences between brains and von Neumann computers is scale integration, which dissolves this hardware software distinction. Is it relevant to cognitive function? And now you've got a property which is fundamentally at odds with a computational function.

It’s almost that the particular urgency of the moment, the prominence of this question, is giving the issue of “what kind of thing is the brain” a time in the sun that it wouldn’t otherwise have. And so we’ve got an opportunity here to finally go beyond the prevailing metaphor that we’ve used in neuroscience for a long time, which is the computational metaphor. We’ll still have a metaphor. It will still be a model. But I think it’s it’s the right time to question our current metaphors and find a better one.

Most helpful sources

Birch, J. (2026). ‘AI Consciousness: A Centrist Manifesto.’ Pre-print.

Block, N. (2026). ‘Can only meat machines be conscious?’ Trends in Cognitive Sciences, 30:4.

Butlin, P. et al. (2023), ‘Consciousness in Artificial Intelligence: Insights from the Science of Consciousness,’ Artificial Intelligence.

Goldstein, S. and Lederman, H. (2026). ‘AI Death,’ Philosophical Perspectives.

Kuhn, R. L. (2024). ‘A landscape of consciousness: Toward a taxonomy of explanations and implications,’ Progress in Biophysics and Molecular Biology, 190.

Seth, A. and Bayne, T. (2022). ‘Theories of consciousness,’ Nature Reviews Neuroscience, 23.

Seth, A. (2025). ‘Conscious artificial intelligence and biological naturalism,’ Behavioural and Brain Sciences.

Schneider, S. et al. (2026), ‘Is AI Conscious? A Primer on the Myths and Confusions Driving the Debate,’ Philosophy and the Mind Sciences, v. 1.