AI Safety Fears Resurface After Anthropic Researcher’s Departure
Qwenews.com – Concerns about the pace of advanced artificial intelligence development have again moved into public view after Jacob Coxon, a 27-year-old former Anthropic researcher, used a resignation thread on X to warn that the industry may be taking unacceptable risks.
“The people building AI earnestly believe that it could kill us all by the end of the decade.”
Coxon described future systems as potentially superhuman tools capable of breaching digital defenses, transforming fields at extraordinary speed and gaining access to real-world resources. His central accusation was that companies are pursuing self-improving superintelligence without a sufficiently credible path to keep it under human control.
“They are racing straight to self-improving superintelligence and gambling with our lives.”
A familiar divide within the AI industry
The dispute reflects a longstanding tension inside the artificial intelligence sector. Anthropic itself was founded after employees left OpenAI amid disagreement over how aggressively the technology should be developed and how much weight should be placed on safety measures. Its founders argued for a more cautious approach to increasingly capable AI systems.
Yet Coxon’s departure suggests that even organizations created around safety-focused ambitions face internal disagreement about whether existing safeguards can match the speed of technical progress. Similar objections have emerged from employees across the sector over recent years, with researchers and engineers publicly questioning whether companies can reliably control systems that become far more capable than their creators.
Evan Hubinger, a more senior Anthropic employee, responded to Coxon’s remarks on X with an unusually direct statement of support.
“We really do earnestly believe AI could kill all humans!”
Hubinger later clarified his position, saying he personally places the likelihood of such an outcome below 10% during the next decade. He also said Anthropic, despite its intentions, does not yet have a plan that guarantees safety if AI reaches a level beyond humanity’s ability to control it.
What researchers mean by superintelligence
Superintelligence is generally used to describe a theoretical point at which an AI system exceeds human performance across a broad range of intellectual tasks. There is no universally accepted benchmark for that milestone. Some researchers argue that elements of it may already be appearing in narrow areas, while others doubt that a broadly superior machine intelligence will ever exist.
The uncertainty is important. Predictions about when advanced AI might arrive vary sharply, but the debate over safety is not limited to a distant hypothetical future. Researchers are also concerned with present-day failures: unreliable outputs, cybersecurity risks, misuse by malicious actors and the challenge of evaluating models before they are broadly deployed.
Anthropic has said its public risk reporting recognizes severe long-term possibilities while assessing current systems as having a low probability of independently acquiring that kind of power. A company spokesperson said the organization has consistently acknowledged both the promise and dangers of AI and is building what it describes as some of the industry’s strongest protections.
“We have always been transparent that AI will bring both enormous benefits and unprecedented risks.”
Warnings from inside leading labs
Anthropic CEO Dario Amodei has repeatedly cautioned that competitive pressure could lead to a catastrophic human error, including a situation in which a company loses control of its AI. In February, he described the problem as a difficult engineering challenge and said that something could go wrong with an AI system developed by one of the companies in the race.
“This is a complex engineering problem and I think something will go wrong with someone’s AI system. Hopefully not ours.”
OpenAI Chief Scientist Jakub Pachocki issued a related warning two days before Coxon’s post, saying AI capabilities are progressing more rapidly than researchers’ ability to monitor and control them reliably. The remarks underscore the growing concern that technical performance may advance faster than the methods used to test, govern and contain advanced models.
In July, nearly 1,400 employees from AI companies signed an open letter calling on the US government to regulate the sector, curb the influence of major technology firms and slow development enough to address safety issues. The message was part of a wider push for clearer rules before highly capable systems become more widespread.
Regulation remains unsettled
Despite the increasingly visible warnings, federal action in the United States remains limited. The Trump administration has sought to weaken state-level AI rules, while Congress has not adopted broad legislation governing the technology. A common argument against stronger US restrictions is that they could benefit China, where companies might face fewer constraints.
China, however, has already introduced several AI regulations, especially around risk management and safety. Last year, the government required companies to label AI-generated material to make it easier to identify and trace. Those rules do not amount to the stricter limits sought by some Silicon Valley researchers, but they challenge the idea that no regulatory framework exists elsewhere.
For now, US AI companies largely remain responsible for policing their own systems. The White House has moved toward a voluntary process for reviewing certain models before release. Representatives from OpenAI, Anthropic, Google and Meta took part in discussions with the administration last month.
Details of that framework are not expected to be fully public, and a June executive order indicated that many of its standards would be classified. That approach leaves consumers, policymakers and independent experts with limited visibility into how the most powerful AI products are assessed before launch.
Coxon’s resignation does not resolve the debate over whether superintelligence is imminent or even possible. It does, however, reinforce a difficult question for the industry: if leading researchers believe the stakes could be existential, how much evidence of safety should be required before companies move faster?
Related Reading
Frequently Asked Questions
What is Gambling with our lives?
Gambling with our lives is the main topic of this guide. The article explains the context, practical details, and next steps readers should understand.
Why does Gambling with our lives matter?
Gambling with our lives matters because readers are looking for a useful answer, not just a short summary. Good content should match search intent and help them decide what to do next.