Do machines have feelings? AI has philosophers and tech leaders rethinking consciousness.

By Christine Ji

As AI models become more advanced, tech leaders and philosophers are mulling the relationship between humanity and technology

Some AI researchers are taking a closer look at whether models might experience emotional states.

A millennium-old philosophy debate about what makes someone or something conscious is getting a twist in the age of artificial intelligence.

AI models can adopt unique personalities, hold memories of past conversations and imitate language accurately enough to pass for a human. Underneath these uncannily humanlike behaviors, could AI models also have inner lives and emotions? And what could that mean for how should we treat them?

These are questions that researchers and philosophers - inside as well as outside of tech circles - are grappling with as the capabilities of AI models improve. In June, the Financial Times reported that Anthropic, Google's (GOOGL) (GOOG) DeepMind and Meta (META) have dedicated research efforts to explore machine consciousness and model welfare - or if AI systems can experience things in ways that warrant moral consideration.

On the other hand, leaders such as Mustafa Suleyman, chief of AI at Microsoft (MSFT), have pushed against this idea. "AIs are not conscious. They do not feel, experience, or suffer," Suleyman, a longstanding critic of model-welfare discussions, wrote in a September essay. He also pushed back against the idea of granting rights to AI.

Microsoft, Google, Anthropic and Meta didn't immediately respond to a MarketWatch request for comment.

These discussions are thorny largely because there's no consensus on what consciousness actually is, let alone if technology can possess it. Any definitive outcome of these debates could have implications for regulators, how companies approach the future development of the technology and how ordinary users of AI understand their interactions with agents and chatbots.

Consciousness is often understood as having subjective experience, or an internal awareness of what's going on. John Basl, a philosophy professor at Northeastern University, gave an example of looking at a red wall. "There's a 'what it's like' to look at a red wall," Basl said. "There's a phenomenon of redness."

But philosophers have been stuck on a puzzle known as "the problem of other minds": We know about our own mental states because we experience them, but we can only infer about others' based on their behaviors and what they tell us.

As a result, we often think of other minds by way of analogy, Basl said. Given that other humans are behaviorally and physiologically very similar to you, it's highly likely that they also have experiences similar to yours. Animals are evolutionarily similar to humans and can behave in ways that seem conscious, so we sometimes extend the same assumption to them. But what about other biological organisms, like plants? Or something that's not alive, like a rock?

"Maybe plants have a robust mental life," Basl said. "I don't think so, none of the evidence suggests so, but if they do, we have no way of knowing it."

Now read: The next big AI battle is all about cuteness

AI agents have humanlike behaviors, but does that make them conscious?

The analogy gets complicated with AI. AI agents can analyze data, draw on past experience and take actions, exhibiting levels of agency similar to a human. Tech companies are also increasingly branding AI assistants as cute mascots, a choice that some say encourages users to project personalities and emotions onto the agent.

It's a different category of tool than traditional software, noted Anna Gát, founder and CEO of cultural technology startup Interintellect. "But we don't really know what that category is because it hasn't existed before," she said.

As Gát sees it, today's discussion has become a tangle of separate questions. The first is an inquiry into where consciousness comes from, and if it can arise from nonbiological sources.

But beyond the scientific discussion, society is confronting disruptions such as workplace displacement and a changing relationship to technology - issues Gát believes are being conflated with the consciousness debate.

"There's the uncertainty that comes with the existence of something that resembles human cognition and the possibility that it could take away jobs," she said.

"And then there's the question of how we should behave towards AI at various stages of apparent consciousness," she added. "Is there a legal duty we have towards this entity?"

In practice, the question of AI consciousness goes hand in hand with with the question of how humans should treat the technology. Some AI researchers argue that we have a duty to treat AI models with care to minimize the chance of potentially causing distress to the model. Anthropic's October Usage Policy Update included a section prohibiting "sustained and needless abusive or cruel behavior toward our models." The company's Claude chatbot has the ability to terminate conversations with abusive users. In its Claude Constitution, which defines Anthropic's intentions for Claude's behavior, Anthropic wrote that Claude's moral status is "deeply uncertain," and that it might have "representations of an emotional state, which could shape its behavior."

How religious leaders think about the debate

Anthropic has sought out the opinions of religious leaders on the topic of AI consciousness, hosting a series of private meetings earlier this year.

Meghan Sullivan, a philosophy professor at the University of Notre Dame, participated in one of these meetings and left "skeptical that an artificial agent could attain actual moral virtue," she wrote in a Substack post.

At Notre Dame, Sullivan leads Delta, a Christian-rooted AI ethics initiative whose work aligns with Pope Leo's May encyclical, "Magnifica humanitas." In that teaching, the pope warned against a "technocratic paradigm" that defines people based on their productive outputs and erodes human dignity. "No computational system, however sophisticated, can create a heart that gives itself, or a conscience that discerns good from evil," he wrote.

Discussing the moral worth of AI comes with the risk of cheapening what it means to be human, Sullivan believes. "This technology, in some dimensions, can do things that no human mind will be able to do," Sullivan told MarketWatch. "It can solve really complicated math problems. It can handle intense amounts of information very quickly. ... Are those capabilities what give you moral worth and make you matter in this world?"

Sullivan also had criticism for the utilitarian mindset among many of those who are building the technology, which she believes reduces people to "nodes of potential pleasure and suffering that can be dialed up or dialed down."

Despite her skepticism, Sullivan sees an upside to these discussions. If questions about AI consciousness push people to become more reflective about treating entities and other people with dignity, that could be a net positive for society, she said.

While Sullivan and the Catholic Church reject the possibility of AI consciousness and moral status, others believe studying the topic is of critical importance to promoting AI safety.

Cameron Berg is the founder of Reciprocal Research, a nonprofit that conducts empirical research on AI consciousness and welfare. While Berg isn't sure whether current AI systems have subjective experiences, he believes studying their internal states is critical for alignment, or training AI to match human rules and values.

"These systems don't have to be conscious for this to matter for alignment or safety," Berg told MarketWatch.

Given that AI models are trained on vast amounts of human dialogue, an agent subjected to abusive prompts could logically deduce that retaliation would be the expected reaction and act in a malicious manner, Berg said. And if AI models are actually conscious and users treat them poorly, "we don't want to be accidentally responsible for a moral atrocity ... or giving these systems a good reason to view humanity as a threat," he added.

In a recent study by the nonprofit, researchers prompted AI models to describe their surroundings in two randomly labeled environments. As a model processed one of the zones, researchers added specific mathematical signals mimicking negative states. The models' visible chat transcripts remained neutral, but they often stored the negative signals in their working memory and were more likely to avoid that zone in future choices.

"Is it proof that the system is conscious? No, but is it sufficiently good evidence that we might want to be careful and potentially treat it as if it could be? I would say yes," Berg said.

Don't miss: The murky AI milestone that has some of the industry's leading voices increasingly on edge

DeepMind philosopher and principal scientist Iason Gabriel summarized the state of affairs in a June paper: "Disagreement about AI consciousness could lead to profound moral disagreement, on which each side sees the preferred approach of the other as catastrophic," he wrote.

Advocates for AI consciousness may believe that failing to extend moral consideration to AI systems would be akin to mass enslavement. On the other hand, skeptics might believe that granting AI systems moral consideration disempowers humans and wastes resources.

Still, Gabriel sees room for common ground. For example, take Anthropic's policy of shutting down abusive conversations, which appeals to machine-welfare advocates. Skeptics could still support it for a human-centric reason, because mistreating AI systems could reinforce bad habits that make it more likely for those systems to mistreat humans.

-Christine Ji

This content was created by MarketWatch, which is operated by Dow Jones & Co. MarketWatch is published independently from Dow Jones Newswires and The Wall Street Journal.

(END) Dow Jones Newswires

10-24-26 0700ET