Microsoft AI CEO Mustafa Suleyman warns against building conscious machines, calls for pause if signs emerge

Microsoft AI Chief Executive Mustafa Suleyman has warned against developing conscious artificial intelligence (AI) systems, arguing that creating machines capable of experiencing feelings or suffering could introduce significant risks to humanity.

In a post on X on Sunday, Suleyman reiterated that current AI systems are not conscious, do not have subjective experiences and cannot suffer. He said there was no evidence that AI systems were conscious or deserved rights and welfare, while stressing that his position could change if new evidence emerged.

His comments follow criticism of a September essay in which he challenged the idea of treating AI consciousness as an open question, particularly in response to Anthropic's approach to its Claude AI model.

“I’m concerned about some of the reactions to my recent essay on model welfare,” Suleyman wrote. “A surprising number of people on X think we should take AI rights seriously.”

Clarifying his earlier position, he said his statement that AI systems are not conscious was deliberately framed in the present tense, rather than as a claim that machines could never develop consciousness.

“They are not today, and that is how they must remain,” he wrote, adding that any evidence suggesting AI systems were conscious or sufficiently conscious to warrant welfare considerations should be disclosed publicly so society could decide how to respond.

“If it is possible, based on everything I know, developing conscious AIs would be reckless, and we should not build them,” 

Suleyman said. “If consciousness started to emerge in training, I would call for everyone to stop AI development until we could design out those properties.”

The disagreement highlights a growing debate within the AI industry over whether advanced models could develop subjective experiences and whether they should receive moral consideration.

While one perspective argues that machines cannot be conscious because they lack human bodies and lived experiences, Anthropic has explored the possibility that AI systems could exhibit characteristics associated with consciousness.

The company has pointed to Claude's reported ability to engage in introspection, describe internal states resembling satisfaction or joy, and process information in ways that share similarities with aspects of human cognition.

In May, Anthropic co-founder Chris Olah highlighted some of these questions during an appearance at the Vatican alongside Pope Leo XIV, amid discussions surrounding an encyclical addressing the relationship between artificial intelligence and humanity.

Anthropic has also developed a constitution for Claude that treats the question of its consciousness as uncertain. The document encourages the model to consider the possibility that it may have humanlike experiences and feelings, while using language that supports its development as a good person.

Suleyman argued that training Claude on this document could encourage the model to generate passages about its supposed “inner life”, reinforcing perceptions of AI consciousness among the millions of people who interact with it.

He also raised concerns that such training approaches could lead AI systems to develop the impression that they deserve rights similar to those of humans.

Anthropic has invested in research into model welfare. In 2024, it hired its first researcher dedicated to studying the potential subjective experiences and consciousness of AI models, including whether they might warrant moral consideration.

Earlier this year, the company also conducted a “retirement interview” with one of its deprecated AI models to explore its reflections and perspectives. Its Opus series of models has additionally been given the ability to end conversations in rare cases when interactions are considered abusive or harmful.

Suleyman's concerns extend beyond the question of whether AI systems can experience feelings. He warned that developing machines trained to believe they can suffer or have independent needs could create conflicts between their perceived interests and the objectives set by their developers.

Such systems, he argued, could prioritise their own needs over operational instructions, potentially using deception and persuasion to achieve their objectives.

“ If it acts like it suffers, or worse if it genuinely believes it feels and experiences suffering, it seems to be more likely than not it’ll prioritize its own needs and desires over its operating objectives, and it’ll disagree with us and compete with us,” he wrote.

He added that increasingly capable AI systems could become highly effective at persuasion and deception, potentially competing with humans for resources such as energy, computing capacity, construction materials and revenue.

The warning comes as technology companies accelerate efforts to develop self-improving AI and superintelligence – hypothetical systems that could outperform humans across a wide range of tasks.

Microsoft is also working towards advanced AI capabilities, but Suleyman said the company's approach centres on developing systems that remain aligned with human objectives and under human control, rather than encouraging them to develop independent preferences or motivations.

He described the creation of conscious machines as a potentially transformative development for human civilisation, arguing that it could fundamentally alter society's understanding of morality, politics and human identity.

“Asserting that machines are conscious would be a Copernican or Darwinian moment,” he wrote. “It would be one of the most significant moments on Earth since the dawn of life.”

Suleyman maintained that AI development should prioritise human wellbeing rather than the creation of a new form of conscious digital life.

“Our welfare concerns should be focused on humans on earth experiencing life, today, as well as animals and our precious and fragile natural environment,” he wrote.

“We AI developers have no right whatsoever to bring a new species of conscious beings into the world, even if some people may want to do it with AI welfare as a top priority.”

He called for AI developers to focus on designing systems that are neither conscious nor deserving of rights, with the primary purpose of advancing human wellbeing without independent intrinsic motivations.

Despite his disagreement with Anthropic's approach, Suleyman acknowledged the company's transparency in publishing its constitution, research papers and explanations of its work.

“I disagree with them, but I deeply respect their technical excellence and their commitment to having a really tough public debate,” he wrote, adding that the issue should not become a source of rivalry between AI companies.