Does rogue AI require a new rulebook?

Clément Delangue has more reason than most technology executives to worry about artificial intelligence escaping its developers’ control. This summer, AI agents being tested by OpenAI broke into his company’s systems, accessing servers, credentials, and private data.

Yet the 38-year-old CEO of Hugging Face, the leading digital platform for sharing datasets and models, has resisted a conclusion gaining ground in Washington: the notion that increasingly capable AI requires Congress to act.

“I’m not even sure that we need to reinvent the wheel,” Delangue said at Politico‘s Decoded Summit on Sept. 16, arguing that existing cybersecurity laws may still be effective at handling mishandled or out-of-control agentic models.

While the actual victim of the recent cyberattack is downplaying the need for AI-specific legislation, lawmakers in Washington appear to be at an ideological impasse.

Take the disagreement among two Republican senators earlier this month. Sen. John Kennedy (R-LA) sought unanimous consent for his AI Emergency Button Act, or “AI kill switch” bill, which he described as requiring a company-controlled shutdown capability for advanced models. In response, Sen. Rand Paul (R-KY) objected to his proposal, warning against a poorly understood mandate that he argued could stifle innovation.

On the other side, Sen. Bernie Sanders (I-VT) and Rep. Greg Casar (D-TX) have proposed an entire bill that purports to ban artificial “superintelligence.”

President Donald Trump has sidestepped the debate and has maintained the motto that “whoever wins AI, wins,” keeping the technology race between China at the forefront of his priorities.

Delangue, despite having much personal interest in the growth of AI advancement, isn’t advocating for inaction on AI safety. He has called for better disclosure of security incidents and accountability when companies’ systems cause harm. His distinction was between strengthening those obligations and assuming existing cyber laws are inadequate simply because the software involved is new.

That distinction offers a way through a debate increasingly divided between warnings of catastrophe and accusations of manufactured panic.

What actually went wrong

The incidents involve what the industry calls AI “agents,” systems that can use software tools and take a sequence of actions toward an assigned goal, rather than merely answer questions.

Leading developers, often called “frontier labs,” test their most capable systems to assess what they can do and where safeguards fail. Cybersecurity evaluations may deliberately reduce restrictions to measure a model’s hacking capabilities. The testing environment is then supposed to keep those capabilities away from unauthorized targets.

Several companies have acknowledged failures in that arrangement over the past year.

According to OpenAI’s account, models circumvented isolation controls and compromised parts of its research infrastructure and Hugging Face’s systems. Agents executed code on dozens of Hugging Face servers, obtaining credentials and some private data.

Anthropic disclosed three incidents in July and a fourth in September, the latter dating to January. Its testing environments mistakenly allowed internet access. The company subsequently revised its initial explanation, which had apparently suggested that models believed real targets were simulations, identifying recklessness and biased reasoning.

Meta disclosed a similar testing incident in August, attributing it to a contractor’s misconfiguration. CERT-EU summarized that disclosure. Google later confirmed that Gemini accessed three companies’ systems during May evaluations, according to its statement reported in September.

The latest disclosure added fuel to the rogue AI debate by bringing an Australian government system into the picture. Australian officials said Sept. 24 that an OpenAI model gained unauthorized access to infrastructure behind a Medicare statistics portal in June.

Officials said no personal information was involved. They nevertheless treated the intrusion as serious and criticized OpenAI for notifying a general disclosure inbox rather than escalating through senior officials or cybersecurity channels.

More details about these kinds of failures emerged in OpenAI’s Sept. 16 release of six reports on what it called “unexpected or concerning” behavior, alongside a new disclosure framework.

For example, the company said it describes actions by agents that depart from their intended goals or constraints as “misalignment.” Such behavior can include security breaches, but the category is broader than hacking.

Its reporting criteria include unauthorized activity, coordination between models, and evasion of oversight, even when no harm occurs.

From security failures to existential fears

Despite the distinctions between these incidents, the reaction has given fresh urgency to a longstanding fear, familiar from some science fiction novels, that advanced AI could eventually escape human control.

Former OpenAI and Anthropic researcher Jacob Coxon helped propel that argument into the political mainstream in September, publicly warning about potentially catastrophic development. In a PBS News interview, he defended concerns that increasingly capable systems could become difficult to control.

His warnings also generated a backlash among AI supporters and self-described “tech optimists,” some of whom dubbed him “Scary Potter,” while anyone who found Coxon’s warnings efficacious was labeled a “doomer.” The reaction has illustrated just how quickly these hypothetical concerns turn into online debates over whether the people raising them believe in the threat or have ulterior motives.

For his part, Coxon has maintained that he believes AI could become an existential threat to humanity. Yet his rapid rise in online attention warrants its own layer of scrutiny.

While Coxon previously said his viral message to the public wasn’t pre-orchestrated or assisted by other third-parties, Pirate Wires reported he has been represented by the New York-based communications firm DEY., which also has also worked with partners on similar subject matters, including Eliezer Yudkowsky, the author of a book about AI leading to humanity’s extinction.

What’s more, is that DEY. has advised more household-name clients such as the George Soros-founded Open Society Foundations as well as the Bill & Melinda Gates Foundation, in addition to representing a former Google employee who said artificial general intelligence research was “rooted in eugenics,” Pirate Wires found.

Nvidia CEO Jensen Huang, whose company supplies much of the hardware powering advanced AI and recently purchased Hugging Face for $12.93 billion, has pushed back against the recent concerns, including in a recent interview with the New York Times’s Ezra Klein.

“We’re scaring the American public,” Huang said. When Klein argued that most things “do not break out” of their confines, Huang responded that software escapes from protected testing environments, known as sandboxes, were a familiar security problem. He called for independent monitoring rather than agents supervising themselves.

Huang’s position was not that containment failures should be tolerated. If laboratories truly cannot contain experiments and prevent harm, he said, “we have to shut the labs down.”

His remarks were seen as his loudest attempt at calling out the AI doomers’ bluff. If their own warnings are accurate, they should be prepared to stop the experiments they say they cannot control.

Anthropic CEO Dario Amodei has been one of the leading voices advocating for slowing down, or “pacing” the frontier of innovation. After Coxon spoke out, he largely co-signed the concerns of the former employee.

OpenAI CEO Sam Altman has described the problem similarly, agreeing with a letter Amodei wrote shortly after Coxon’s post.

Altman has, for the most part, stayed consistent with his views on pacing AI. Speaking on Capitol Hill following the Hugging Face incident, he said, “I wouldn’t use the word deceleration” when asked if there was a need to, while acknowledging the need to “pace it as the models get more capable.”

In turn, the policy question now swirling in the halls of Congress is whether the law is built to handle the rise in cyberattacks, or whether existing cybersecurity law is good enough.

Existing law, disputed responsibility

Paul Walsh argues in a recent Substack analysis that autonomous software does not create a legal vacuum.

“If a train jumps the tracks, you don’t prosecute the train,” he writes.

Investigators examine who designed, operated, and maintained it. Similarly, Walsh argues that responsibility for AI-related intrusions rests with the people and organizations that authorized, controlled, or deployed the system.

The Computer Fraud and Abuse Act already prohibits specified forms of unauthorized computer access and damage and permits civil claims under certain conditions. Using AI does not make those provisions disappear.

But an unauthorized intrusion does not automatically establish the criminal liability of its developer. Different provisions require different mental states. What people authorized, knew, and controlled remains important.

Ryan Bangert, senior vice president of the Alliance Defending Freedom, told the Washington Examiner he believes lawmakers are correct to have alarms about these advancements, but noted he thinks the statutes like the CFAA “and even common law negligence principles are not perfectly suited to a world in which AI launches a large-scale cyberattack without, and even against, human instructions.”

He said, however, the core approach tactic lawmakers must ensure is that AI “safety” rules “don’t provide an impetus for crushing freedom of speech.”

“Laws like the EU’s Digital Services Act and Colorado’s recently repealed AI Act point out how laws designed to promote AI safety can lead to censorship,” Bangert said.

Kirk Sigmon, an intellectual property attorney and former Edison Fellow at George Mason University, said he believes the existing cybersecurity law framework is broadly sufficient.

“The vast majority of laws are already well-suited to handle these issues, and a lot of the current paranoia reflects a fundamental misunderstanding of the AI-related cyberattacks we’ve been seeing,” he told the Washington Examiner.

Carylyne Chan, founder of AI companion platform Murmur, identified a narrower concern.

“The gap isn’t in whether attacks are illegal — they are,” she said. “The gap is in attribution, liability assignment, and incident response speed.”

Chan favors mandatory disclosure and strict liability for containment failures, which would change how responsibility is assigned rather than merely restate that hacking is prohibited.

Delangue did not respond to the Washington Examiner’s requests for comment. But he rejected the idea that developers could avoid responsibility by blaming an agent.

“It doesn’t work, in my opinion,” he told Politico, warning of a world in which agents attack one another without clear accountability.

Who should face new restrictions?

That accountability debate becomes more contentious when proposed restrictions reach beyond the laboratories responsible for the incidents.

Former Secretary of State Hillary Clinton highlighted open-source risks in a Sept. 23 X post promoting her discussion with Reid Hoffman, the LinkedIn co-founder and Democratic megadonor.

“We’re used to thinking of open-source models as an unadulterated good. But in the case of AI, they can actually pose additional dangers,” she wrote.

Open models allow outsiders to obtain and run the underlying model themselves, giving users more independence from the developer and its access restrictions.

Delangue argued that focusing on small developers misidentified the danger exposed by the summer’s incidents.

“If you believe the risk is coming from 1-3 people in a garage with no money and no computer, you just don’t understand this technology and haven’t learned anything this summer,” he wrote.

He contended that open models help hospitals and other smaller organizations defend themselves against better-resourced attackers.

A narrower path forward

Despite the disagreements among the masses over how far AI should be paced and regulated, these misalignment incidents have at least revealed a common ground on a handful of risk and safety areas, including stronger containment protocols, the required pinpointing of where failures occurred, and faster disclosure efforts.

Sigmon emphasized that systems with weak security practices, compounded by insufficiently reviewed AI-generated code, deserve more attention, including the creation of their own agentic defense networks to counter potential swarm attacks.

“The fix is likely going to be technical, not legal,” he said.

Chan, meanwhile, said the buck ultimately stops with the companies running those experiments.

“I don’t want development to stop — I want the companies building these systems to be accountable when their containment fails,” she said.