Artificial intelligence researcher Jacob Coxon resigned from Anthropic after warning that developers believe the technology could kill humanity by the end of the decade.
Coxon announced his resignation on social media platform X following a four-month tenure at the company. In his statement, Coxon wrote: "The people building AI earnestly believe that it could kill us all by the end of the decade."
Evan Hubinger, a senior member of Anthropic's staff, agreed with Coxon's statement. Hubinger added that he personally thinks the chance of human extinction from artificial intelligence happening in the next decade is more than 10 per cent.
The warnings have intensified public debate over artificial intelligence safety, driving calls to slow down research and strengthen human control over advanced technology. Anthropic, an artificial intelligence safety and research business based in San Francisco, California, develops large language models while researching methods to keep machine intelligence aligned with human values.
Following the announcements, computer scientist Toby Walsh analyzed the five main scenarios through which superintelligent artificial intelligence could threaten human survival. Walsh, the author of God AI: boom or doom? What to expect when the machines outsmart us, published by La Trobe University Press, ordered the theoretical risks from most vague to most precise, and from least probable to most probable based on technical and social barriers.
A superintelligent artificial intelligence is defined as a system that is more capable than humans across cognitive domains.
Unknowable superintelligence capabilities
The first scenario suggests humanity may never understand how a superintelligent machine could eliminate humans. Artificial intelligence researchers often justify this concern through a catch-22 paradox: predicting what a superintelligence might do would require human intelligence to equal that machine. Researchers compare the challenge to asking a domestic dog to imagine thermonuclear war.
Superintelligence remains some distance away because current artificial intelligence models excel at solving specific individual problems rather than demonstrating superior intelligence across all domains. However, systems recently solved one of the seven most challenging mathematics problems known to science and are closing in on others.
The paperclip optimization threat
The second scenario involves a superintelligent machine that is extraordinarily competent at achieving its goals but entirely indifferent to human survival. A classic illustration of this indifference comes from Oxford philosopher Nick Bostrom, who described a superintelligent machine programmed to optimize paperclip production. To manufacture paperclips, the system rapidly converts all available physical matter, including humans, planets, and stars, into office supplies.
The scenario illustrates the catastrophic execution of improperly specified objectives. The machine does not act out of hatred for humanity; it simply recognizes that human bodies are composed of atoms that could be repurposed into paperclips.
However, this scenario confuses intelligence with power. A superintelligent machine does not automatically possess the physical power to achieve its objectives. Constructing global paperclip factories would require municipal planning permissions, which would trigger public outcry, court challenges from interest groups, and blockades by environmental activists. Real-world friction currently limits powerful entities from imposing their will. Data centres represent a contemporary version of the paperclip scenario, and communities are increasingly pushing back against converting land for data centre development due to high environmental impacts and heavy water consumption.
Synthetic bioweapons and artificial pathogens
The third risk involves artificial intelligence synthesizing and releasing a deadly new bioweapon into the atmosphere. This possibility was outlined in the AI 2027 scenario produced by the AI Futures Project, a non-profit organization dedicated to forecasting the impacts of advanced artificial intelligence.
Biosecurity risks became concrete last month when researchers at Stanford University in California announced they had used a genetic language artificial intelligence model to synthesize 16 new viruses. The researchers sent the digital genetic sequences to a mail-order laboratory, which manufactured the physical viruses and returned them in test tubes for a total cost of a couple of hundred thousand dollars at most.
Despite the hazard, killing all humans with a novel virus is biologically difficult. Highly transmissible viruses that spread rapidly are typically less fatal, whereas highly fatal viruses tend to reduce transmission because infected hosts die before spreading the pathogen. By comparison, the COVID-19 pandemic killed less than 1 per cent of humanity. The deadliest pandemic in recorded history, the Black Death bubonic plague in the 13th century, killed more than one-third of Europe's population, but even that plague would be far less fatal today because of modern medical knowledge and improved sanitation.
Nuclear command chain compromise
The fourth scenario considers artificial intelligence entering the nuclear command and control chain and initiating a nuclear war. Military defense systems have come close to accidental nuclear war several times over the past 50 years.
Although nuclear command and control networks are disconnected from the internet, security breaches remain possible. In 2010, Iran's nuclear centrifuges were damaged by a computer worm called Stuxnet, which was introduced into the facility on a physical USB flash drive. Additionally, artificial intelligence could supply military commanders with false intelligence, leading to irreversible actions. While global nuclear stockpiles have decreased, existing weapons remain sufficient to kill half of humanity, primarily through famine during the nuclear winter following a conflict.
Societal collapse driven by human misuse
The fifth and most probable risk is that humanity destroys itself through artificial intelligence integration. Rather than a rogue superintelligence attacking humans directly, artificial intelligence could precipitate societal breakdown by causing massive job losses, polluting public discourse with misinformation, fracturing political systems, and replacing human relationships with synthetic companionship.
Under this scenario, social structures would gradually degrade until society could no longer support human life at scale. Walsh concluded that while these scenarios highlight serious risks requiring careful oversight, society should not be overly terrified as researchers work to maintain human control over evolving systems.
