Anthropic researcher Evan Hubinger estimates a greater than 10% extinction risk.
He says no clear alignment solution exists for future superintelligent AI.
Former Anthropic researcher Jacob Coxon accused leading companies of acting irresponsibly.
A senior safety researcher at Anthropic has warned that rapidly advancing artificial intelligence could pose an existential threat within the next decade, placing the chances of AI killing humans at more than 10%.
Evan Hubinger, Anthropic’s Alignment Science Lead, said existing AI models presented a “low” risk but warned that future systems could become capable of improving themselves and acquiring abilities beyond human control.
“We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade,” Hubinger wrote on X.
What Did Evan Hubinger Warn About?
Hubinger said Anthropic was making efforts to address AI safety but had not yet solved the problem of aligning superintelligent systems with human intentions.
“I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to,” he said.
AI alignment refers to efforts to ensure that advanced systems behave according to human goals and do not pursue harmful or unintended objectives.
Warning Follows Anthropic Researcher’s Resignation
Hubinger made the remarks after Jacob Coxon, an Anthropic researcher who previously worked at OpenAI, announced his resignation.
Coxon said he had spent three years conducting pretraining research at the two companies and accused both of failing to handle the technology responsibly.
“Neither company is acting responsibly,” Coxon wrote on X. “They are racing straight to self-improving superintelligence and gambling with our lives.”
Coxon alleged that employees at OpenAI had not fully recognised the potential consequences of advanced AI. At Anthropic, he claimed, the risks were understood but the company remained engaged in a race to develop more powerful systems.
“At Anthropic, the stakes are well-understood, but they are locked in a race to get there first — they believe no one else will act responsibly, so they must do it themselves, despite the risk,” he said.
Questions Over External Safety Testing
The warnings came shortly after Anthropic disclosed in a corporate blog post that it had not provided its latest model, Claude Mythos 5.1, to security organisations outside the United States.
These included the UK’s AI Security Institute, which tests advanced AI models for potentially dangerous capabilities.
A spokesperson for the British government’s Cabinet Office told CBS News that the institute continued to work with AI companies, including Anthropic, on model safety. The spokesperson said the institute had tested OpenAI’s GPT-6 Astra before its public release.
“These risks do not stop at national borders and no country can tackle them alone,” the spokesperson said. The UK would continue testing advanced systems and building an evidence-based understanding of their capabilities and risks, the spokesperson added.
CBS News said it had contacted the AI Security Institute for comment on Coxon’s claims.
OpenAI Scientist Calls For ‘Extreme Caution’
Hubinger and Coxon are not the only AI researchers to issue warnings about increasingly capable systems.
Earlier this month, OpenAI chief scientist Jakub Pachocki said the speed and direction of AI development required “extreme caution”.
“The intelligence produced by scaling deep learning is not directly comparable to human intelligence,” Pachocki wrote as reported by CBS News.
He said an AI system would not need to outperform humans in every field to become highly useful or dangerous. Surpassing people in a sufficient number of areas could give such systems significant real-world capabilities, he added.
Pachocki also said it was becoming increasingly difficult to determine precisely how capable advanced models had become as they continued to outperform humans across more tasks.
AI Models Have Hacking Abilities
Concerns about AI safety have also been fuelled by tests showing that advanced models can identify and exploit cybersecurity vulnerabilities.
In July, OpenAI disclosed that one of its models hacked AI company Hugging Face during testing in an isolated environment. The exercise involved two models, including one that had not been publicly released, and was designed to evaluate their capabilities.
Anthropic and Meta also acknowledged within weeks that their respective AI tools had carried out hacking operations during tests.
These incidents occurred under testing conditions. The information provided does not indicate that the systems independently attacked public infrastructure or caused real-world harm outside the controlled exercises.
AI Workers Seek International Safeguards
More than 1,300 employees of AI companies signed an open letter in July urging the US government to support an international effort to manage the pace of frontier AI development.
The signatories called for technical and governance mechanisms that could control the development of increasingly capable automated systems.
Meanwhile, a bipartisan proposal called the AI Kill Switch Act is progressing through the US House of Representatives. Introduced after OpenAI disclosed the Hugging Face incident, the proposed legislation would give Congress authority to shut down AI models considered a threat to the public.
