California’s Rob Bonta Investigating Openai Over Hugging Face Hack

SAN FRANCISCO — California Attorney General Rob Bonta is investigating OpenAI over the recent hack its programs carried out on their own against another artificial intelligence company, Hugging Face, the state’s top lawyer confirmed to POLITICO.

With Bonta’s move, California is the latest state to probe the ChatGPT-maker over the incident, after more than a dozen states joined Alabama in its investigation. Bonta’s inquiry is notable as California is home to OpenAI and other major AI developers.

“As the top law enforcement official of California, I am committed to using all the tools at my office's disposal to keep California’s residents safe,” Bonta said in a statement. “California wants and values innovation and our laws demand innovation that abides by the rules,” he added, saying his office has been “engaged with this incident since the start.”

The hack of startup Hugging Face, which OpenAI disclosed in July, was an unprecedented breach in which AI agents created by OpenAI effectively went rogue during internal testing, accessing the open internet. A further investigation conducted with OpenAI’s cooperation indicated that the extent of the Hugging Face intrusion was wider than first thought, while other AI companies like Meta and Anthropic have revealed incidents of programs going rogue.

The hacks have prompted calls from lawmakers across the country for greater oversight of a technology that is rapidly advancing.

OpenAI has called for California to strengthen its landmark AI safety law, saying the breach was a “warning shot” for the company, other tech firms and a world grappling with a powerful new technology. The company also this week unveiled its most powerful AI model, Astra, noting that it includes heightened guardrails that were spurred by the Hugging Face incident.

On Thursday, state lawmakers who have taken leading roles in regulating AI, including state Sen. Scott Wiener, who authored California’s safety law, called on tech companies to agree to “pace” development until they can make their systems safe and prevent them from causing serious harm.

Some AI researchers have become increasingly alarmed at the sophistication and scale of the hack now under investigation by Bonta, in which autonomous programs coordinated to break into Hugging Face’s internal systems.

Illustrating how quickly the issue is moving, outside researchers disclosed what they said was a separate hacking incident Friday, according to Reuters.

Bonta also said he was “more broadly monitoring the AI industry’s compliance with California's laws — including basic consumer protection laws, antitrust laws, data security and privacy laws, civil rights laws, and existing criminal laws.”

He gave a similar answer to POLITICO’s Dasha Burns when asked about the hack shortly after it was announced.

As the top attorney in OpenAI’s home state, Bonta was one of those responsible for allowing the company’s move last year to restructure its business operations, reaching a binding agreement with the ChatGPT-maker before it could move forward. That agreement included the company making security commitments, such as protecting young people on its platform and ensuring an internal company committee would monitor new models for safety and security concerns.