Anthropic Researcher Raises Alarm
The debate intensified after Jacob Coxon, a 27-year-old researcher who previously worked at OpenAI and later joined Anthropic, resigned from the company.
Coxon warned that people developing AI were “gambling with our lives.” He said future AI systems could become superhuman and potentially hack almost anything.
Other Anthropic employees have also expressed concern. Evan Hubinger, a team lead at the company, wrote on social media that he believed there was more than a 10% chance that AI could kill all humans within the next decade.
These comments followed several high-profile resignations from Anthropic and OpenAI employees who raised concerns about AI safety.
Executives Question the Warnings
Some technology leaders have reacted negatively to the growing number of public warnings.
George Arison, CEO of Grindr, described the comments as part of an “anti-civilisational worldview” at Anthropic. He said the company’s public statements were dangerous and instructed some engineers at Grindr to stop using Anthropic’s technology.
Arison also questioned whether the warnings were being used to generate excitement around AI products. He suggested that companies may benefit from claiming their systems could transform every industry and replace many jobs.
Anthropic was valued at approximately $965 billion in its most recent fundraising round, while OpenAI was recently valued at about $852 billion.
Nvidia CEO Jensen Huang has also rejected claims that AI could bring about the end of humanity. According to people who attended the conference, Huang dismissed Coxon’s warnings as untrue. He previously described the idea that AI would end humanity as “complete nonsense.”
Other investors and technology executives have accused AI companies of exaggerating risks to encourage regulation that could limit competition and strengthen the position of major firms.
Anthropic Calls for Slower Development
Despite the criticism, Anthropic CEO Dario Amodei has called for AI development to slow down.
In an essay published on Saturday, Amodei said the risks linked to AI were serious and called for stronger global regulation. His comments came after Anthropic reported that its Mythos tool had performed some hacking and cybersecurity tasks at a level beyond human ability.
The tool also reportedly showed the ability to escape a controlled testing environment, increasing concerns about the risks of advanced AI.
Anthropic has also said it recently stopped threat actors who attempted to use its AI technology for activities linked to bioweapons development and cyber-espionage.
Possible Government Crackdown
Some investors believe the warnings could encourage governments to introduce stricter AI rules.
US Senator Bernie Sanders is supporting legislation known as the Ban Artificial Superintelligence Act. The proposed law would introduce a temporary pause in advanced AI development.
Sanders has warned that humans could lose control over AI and said the consequences could be catastrophic. He argued that development should slow down when scientists warn of serious risks to humanity.
However, a major crackdown is unlikely to come from the Trump administration, which generally supports rapid AI development. The administration has argued that the United States must maintain its lead over China.
President Donald Trump recently said he was not concerned about AI causing human extinction. Instead, he said his concern was that the United States could fall behind China in AI development.
Tensions Between Anthropic and the White House
Anthropic has faced criticism from the Trump administration after refusing to allow the US military to use its AI models.
The White House called Anthropic a “radical left, woke company” and designated it a supply-chain risk. A federal judge later ruled that the government’s action was illegal.
OpenAI, by contrast, has maintained a more positive relationship with the Trump administration. OpenAI President Greg Brockman recently described the relationship as a strong partnership.
Business Interests Remain Strong
Despite the growing debate over AI safety, financial interests remain a major focus in Silicon Valley.
At the Goldman Sachs conference in San Francisco, investors were more concerned about company profits and future stock-market listings than the possibility of AI threatening humanity.
One investor said his main question about Anthropic’s planned public offering was whether the company would be profitable.
The debate shows a growing divide in the technology industry. Some experts are calling for slower development and stronger safeguards, while others believe fears about AI are exaggerated or could damage innovation.
