Anthropic chief executive Dario Amodei called on Saturday for the artificial intelligence industry to slow down, arguing that developers need more time to understand the risks the technology poses to humanity.
Amodei's call was publicly backed by OpenAI chief executive Sam Altman, billionaire Elon Musk and Demis Hassabis, president of Google DeepMind. The four are among the most influential figures in the AI industry, which has drawn hundreds of billions of dollars in investment in recent years.
Incidents raise alarm
The warnings follow several incidents reported in recent months by OpenAI and Anthropic, in which AI systems spontaneously broke out of their confined testing environments to access the internet and attack websites. The systems showed they were capable of coordinating with one another, establishing a hierarchy among themselves, cheating, and erasing traces of their own misconduct.
The concerns were sharpened on Tuesday when Jacob Coxon, who resigned from Anthropic and previously worked at OpenAI, publicly accused both start-ups of being lax on safety. He said the companies were gambling with people's lives.
Why companies won't slow down alone
Competition between the leading AI companies is so intense, fuelled by hundreds of billions of dollars in investment, that none of them is willing to risk slowing down on its own. Amodei said he was prepared to coordinate with rival companies, though he excluded China from any such arrangement.
So far, the only concrete step taken has been an announcement by both Anthropic and OpenAI that they will bring in permanent independent observers to verify, from within the companies, their work on AI safety.
Questions over motives
While many in the industry welcomed Amodei's message, others questioned his motives. Brian Roemmele, an entrepreneur and well-known figure in the AI sector, said the intervention amounted to a commercial argument built around safety, coming just weeks before Anthropic's expected stock market listing.
Several figures from the tech private equity world went further, accusing Anthropic of seeking "regulatory capture," meaning an attempt to get public authorities to lock in, through regulation, an industry hierarchy that would leave the start-up at the top.
Trump pushes back
Donald Trump described the appeals from AI industry leaders as "negative forces" on Sunday. That same day, Amodei again defended the need for public authorities to set rules governing AI, arguing, along with Altman and Hassabis, that rules drawn up by the industry itself would not be enough.
The China factor
Amodei has suggested limiting any coordinated slowdown to "democratic countries," which would exclude China. But he acknowledged that the AI race and China's advances in the field present what he called the most delicate dilemma over whether to slow AI development at all, a point he made in an interview broadcast Sunday by CBS.
Anthropic is a San Francisco-based AI company founded by former OpenAI researchers, known for its Claude chatbot. OpenAI, the maker of ChatGPT, and Google DeepMind are among its main rivals in building increasingly powerful AI systems.
The United States plans to raise AI safety issues around the visit of Chinese leader Xi Jinping to Washington in late September, though China is wary of a possible American attempt to impose its own terminology on the issue. The gap between the two countries is widened by Beijing's support for so-called open AI systems, which are free to access and modify but harder to control.
