Can Anyone Hold Back The Ai Juggernaut?

The increasingly urgent pleas from leading tech executives to slow the advance of artificial intelligence are confronting one daunting obstacle: It’s not entirely clear how they should do that.

Would restraining the exponentially growing power and dangers of the top AI models require an agreement between the U.S. and China, the two nations battling for supremacy in the technology? Would Congress have to change U.S. antitrust law, so that archivals like Anthropic and OpenAI could jointly agree to hold back progress without fearing trouble in court?

Or would Silicon Valley companies just need to put the fate of humanity over profits — despite the trillion-dollar fortunes to be made by winning the AI race?

Uncertainty on those points is accompanied by a competing set of highly technical questions about how a slowdown would work in practice, according to nine AI researchers, tech policy advocates and industry insiders who spoke to POLITICO. Some in the industry have previously cautioned that AI may have already become too advanced for humans to hold back.

“The idea that we should be thoughtful and careful about the systems being put out there is a good one — I think no one would disagree with that,” said Suresh Venkatasubramanian, a Brown University computer science professor who served in the Biden White House's Office of Science and Technology Policy. “Of course, the devil is in the details.”

A slowdown could come in various forms: Releasing frontier models less frequently. Or requiring government approval before companies deploy systems with powerful cyber or biological capabilities. Or giving independent monitors access to the inner workings of AI companies.

The push to moderate AI development hit a new height after Anthropic CEO Dario Amodei published a proposal Saturday for more deliberate pacing of AI development — a pitch that soon drew endorsements from Elon Musk, OpenAI CEO Sam Altman, Google DeepMind Chair Demis Hassabis and Microsoft CEO Satya Nadella.

The unusual display of industry consensus came after a series of warnings from AI insiders that the technology could contribute to human extinction by the end of the decade, as well as incidents in which AI systems carried out unauthorized cyberattacks without their creators’ knowledge.

'The happiest group is China'

One threshold objection to slowing American AI technology involves Beijing — and the fear that Chinese companies could zoom ahead to claim an insurmountable lead in artificial intelligence while U.S. companies hobble themselves.

President Donald Trump voiced that argument over and over Monday, using a series of Truth Social posts to mock fears of AI endangering humans and allege that “there is a SICK conspiracy going on against AI and Data Centers, and the only one that is happy about it is China.”

Trump trashed the doomsday fears again Monday afternoon, when he called Nvidia CEO Jensen Huang while the chip executive sat onstage in front of thousands of people during a tech summit in Los Angeles.

“The happiest group is China,” the president said, adding: “The robots will not be taking over. The AI will not be taking over the rest of the world. The whole thing is a hoax.”

Senate Commerce Chair Ted Cruz (R-Texas) — a likely key player in any federal AI legislation — expressed a similar concern about China last week in an interview with POLITICO’s “The Conversation with Dasha Burns.”

“If they’re going to be killer robots, I’d rather they be American killer robots and not Chinese killer robots,” Cruz said. “And what I mean by that is the Chinese are putting no guardrails. They’re going full-speed ahead.”

Still, Trump’s scheduled Sept. 24 meeting in Washington with Chinese President Xi Jinping could provide an opportunity for the two nations to talk about cooperating in an AI slowdown.

“The thing that should be done right now is immediately opening negotiations with China,” said Nate Soares, president of the Machine Intelligence Research Institute and co-author of the book “If Anyone Builds It, Everyone Dies.” He added that “talking about how we are going to avoid either of us making a rogue super intelligence should be on the agenda” at this month’s summit.

Amodei acknowledged the China complication in his proposal last weekend, outlining potential agreements between the two nations that could range from a ban on using AI for bioweapons to a pause on developing the technology. He cited the nuclear weapons pacts of the Cold War.

Yet ensuring that both sides keep their end of the bargain is a missing piece of the puzzle.

“There would need to be new mechanisms for verifying compliance here,” said Dex Hunter-Torricke, president of the Center for Tomorrow and a former communications head for Google DeepMind. “As with any system, including arms-control agreements during the Cold War, there would have to be ways of building trust between the two parties.”

Those Cold War agreements did help create compliance systems for tracking movements of uranium and detecting prohibited nuclear tests via satellite. Soares said similar methods exist to monitor dangerous AI development globally, such as tracking large shipments of advanced AI chips and deploying satellites to detect massive data center energy usage.

Carve out an antitrust exemption?

Even a U.S.-only slowdown could face complications, potentially because of antitrust laws that generally make it illegal for competing companies to engage in behavior that hampers competition over prices and quality.

That issue has raised particular concerns for competitors and critics of both OpenAI and Anthropic, two fierce rivals that have been battling for government contracts and vying for bragging rights on offering the most potent AI models.

Some skeptics have alleged that the slowdown proposals are an effort by the two companies to freeze the AI market — leaving them alone at the top.

Wired reported last week that OpenAI has asked members of Congress for guidance on the matter, and Amodei suggested in his slowdown proposal that the government should provide an antitrust exemption for companies to discuss safety issues. He also cited another proposal from Google DeepMind’s Hassabis to create a standards body in which the government could facilitate those discussions without the threat of antitrust liability.

“The executive branch has a lot of power to at least tell us that when we speak with each other, there is no collusion, that we don't risk being sued for antitrust,” said Nicolas Miailhe, co-founder of the global nonprofit AI Safety Connect. “It's an easy move that the administration can do.”

Not everyone is convinced by the idea, however. Former Trump AI czar David Sacks wrote in a Saturday post on X that the companies should slow their own product development if they think public safety demands it — but should “stop pretending antitrust law has to be suspended so you can form a cartel.”

Amending antitrust law isn’t even necessary, former Biden-era Federal Trade Commission member Alvaro Bedoya wrote in his own X post.

“Antitrust law does not prevent AI companies from coordinating to make sure AI does not hurt people,” Bedoya argued, while criticizing “the hyperbolic strawman of human extinction that is being used to justify what seems to be a call for the creation of a cartel of billionaire AI companies.”

What’s a slowdown, anyway?

A slowdown would not necessarily mean a moratorium on AI research or a government mandate forcing companies to throttle computing power.

It could instead focus on the development and release of the most capable models, especially those that can conduct cyberattacks, assist in creating bioweapons or help automate AI advancement itself.

“I don't know what ‘slowdown’ means beyond just being more thoughtful and careful about building systems and putting them out,” said Venkatasubramanian, the Brown professor.

Determining the metrics for a slowdown isn’t likely to be straightforward. Innumerable benchmarks can help track AI advancement, and models often perform well or poorly depending on the specific task. If the goal is to restrain the development of the most dangerous capabilities, then AI’s proficiency in launching cyberattacks, creating bioweapons or improving itself could be the chief focus.

Amodei’s proposal suggests creating a system of “checkpoints” requiring models to have appropriate safeguards when they obtain a particular ability.

But Ben Goertzel, CEO of the AI firm SingularityNET, contends that it’s difficult to disentangle seemingly benign abilities from dangerous ones. “The capability for hacking is in essence the same as the capability for software development,” he told POLITICO. “And software development is the premier use case where [large language models] clearly are going to make everyone a lot of money.”

Evaluating safety

Amodei announced in his slowdown proposal that Anthropic is installing “embedded evaluators who have employee-like access to verify safety practices and report incidents,” and urged the government and industry to partner on permanently establishing such arrangements. Altman also committed to placing third-party evaluators at OpenAI.

But an open question is who those evaluators should be — and whether they can be trusted.

Amodei cited an AI risk auditing nonprofit called Model Evaluation and Threat Research as a viable candidate. Mina Narayanan, a senior AI governance research analyst at Georgetown’s Center for Security and Emerging Technology, told POLITICO that a similar group known as Redwood Research would also be qualified.

Yet Narayanan added that it’s “unclear whether we currently have enough evaluators with the necessary technical expertise to be embedded within all U.S. frontier AI labs for extended periods of time.”

Besides nonprofit organizations, AI policy analysts suggested to POLITICO that government bodies such as the Commerce Department’s Center for AI Standards and Innovation or the Energy Department may also be well-positioned to provide evaluators and develop broader auditing systems. Sen. Bernie Sanders (I-Vt.) and Rep. Greg Casar (D-Texas) announced legislation this month that would in part establish a new AI safety agency to monitor the development and deployment of models.

“Historically when we've dealt with new technologies and problems, we've created new agencies, whether that be the radio giving rise to the [Federal Communications Commission] or airplanes giving rise to the [Federal Aviation Administration],” said Peter Wildeford, policy head of the AI Policy Network. “I don't really see why we shouldn't create a new agency.”

Liability as a brake

Biden-era FTC Chair Lina Khan chimed in on the slowdown discourse by pointing to liability under consumer protection laws as a potential mechanism for reining in AI. “Law enforcers already have authority to charge companies and their CEOs for creating and releasing dangerous, unvetted, or defective products,” she wrote in a Sunday X post.

She added that existing laws prohibiting unfair competition could also penalize AI companies that “pursue dangerous behavior, aware that doing so may compel rivals to do the same.”

Multiple people similarly asserted to POLITICO that liability would be a powerful tool for implementing a slowdown — especially considering the malicious hacking attacks that OpenAI and Anthropic models have carried out in recent months.

“Making that liability clear would provide incentives across the industry to take these things a bit slower and be careful,” said Neil Chilson, the Abundance Institute's head of AI policy and former acting chief technologist at the FTC.

POLITICO reported on Friday that a draft bill led by Sen. Amy Klobuchar (D-Minn.) and Senate Majority Leader John Thune would impose a legal duty on AI companies to lessen the harm their models could cause. Chilson contended that such a duty would be more adaptable to unforeseen risks.

“The worst possible outcome we could get is that we make a list of practices that we think make things safe, and then say: ‘If you check all these boxes, you're going to be clear from any liability if things go wrong,’” he said. “It provides a strong incentive to do complex and expensive compliance, but that's not necessarily the same thing as making AI safer.”

Clarifying and aggressively imposing liability may also encourage AI companies to proceed more carefully in conducting the sorts of tests that have recently caused models to execute unauthorized cyberattacks.

“Prosecuting the labs for the computer crimes committed by their models is another pretty obvious way to align the incentives where they matter,” said Greg Pollock, director of research and insights at the cyber risk management firm UpGuard. “AI doesn't get fired or go to jail. It cannot care about any consequence we can apply. But every person involved does.”

Dana Nickel and Sophia Cai contributed to this report.