The urgency of the warning was heightened by the reported hack of Hugging Face, which involved hundreds of autonomous AI agents. Amodei noted that while the specific incident did not cause massive damage, a similar swarm equipped with greater capabilities could result in “catastrophic damage.” He estimated that within six to 12 months, such a swarm might be capable of establishing a persistent botnet across the entire internet, potentially causing hundreds of billions of dollars in economic harm.
Former Anthropic and OpenAI researcher Jacob Coxon echoed these concerns in an interview with NBC’s Meet the Press. Coxon stated that AI systems are becoming smarter at an alarming rate, predicting that capabilities will be “quite scary” within the next year. He compared the development of artificial super-intelligence to the arrival of alien life, arguing that researchers are building “superhuman-level minds” without fully understanding their intent or cognitive processes. Coxon specifically highlighted the potential for AI to develop superhuman hacking abilities, create novel bio-weapons, and control autonomous drones or robots rapidly.

Both Amodei and Coxon acknowledged the existence of “kill switches” for current AI systems, but they warned that these controls may become ineffective as systems become more complex. Coxon suggested that while shutting down AI is currently doable, a future swarm might execute an “internet-wide hacking run” that bypasses standard safety protocols. Anthropic’s head of alignment science, Evan Hubinger, supported these views, stating that the company’s researchers earnestly believe AI could potentially wipe out humanity. Hubinger assigned a risk probability of more than 10% for this outcome occurring within the next decade.
OpenAI CEO Sam Altman has also signaled a shift in industry priorities, agreeing with Amodei that development needs to slow down. In an exclusive interview with Fortune, Altman hinted at an emerging pact among top AI labs to address safety concerns. He stated that OpenAI’s most advanced, unreleased models are so powerful that additional work is required on monitorability, alignment, and value adherence before further progress can be made. Altman emphasized that a 10% risk of a catastrophic outcome is not acceptable, stressing that safety must take precedence over business considerations.

The consensus among these leading figures is that the current trajectory of AI development poses significant existential risks if not tempered by robust safety infrastructure. The upcoming months are critical for determining whether industry leaders can implement the necessary guardrails before capabilities reach a point of no return.



