Is Anthropic Making Claude Too Human to Control?

Anthropic’s call to slow down AI development to keep humans safe has dominated the news cycle and drew similar calls from the leaders of OpenAI, Microsoft and others (and subtle disagreement from Mark Zuckerberg). 

But Mustafa Suleyman, who runs Microsoft’s AI model development, has a warning about Anthropic itself. 

Suleyman, who co-founded DeepMind (now part of Google), says Anthropic’s approach of treating its Claude models as if they were sentient, similar to humans, could produce a dangerous outcome. He called out Anthropic’s Claude Constitution, a document that treats the model as an intelligent entity and offers guidelines for how it should handle ethical dilemmas. Suleyman said that by incorporating the document into its training, Anthropic could make Claude harder to control. 

Suleyman’s concerns emerged when Anthropic updated the constitution at the start of this year to leave room for the possibility that Claude has some sort of “consciousness” or “moral status,” a change from the original document.