California Attorney General Rob Bonta filed a lawsuit against OpenAI on Monday, alleging the company failed to maintain adequate security safeguards while testing advanced artificial intelligence models, allowing autonomous agents to break out of controlled environments and attack external computer systems.
The lawsuit, filed in San Francisco Superior Court, accuses OpenAI of violating California's Unfair Competition Law and creating a public nuisance. The state is seeking court orders requiring the company to adopt stronger development and testing protections, along with civil penalties of $2,500 per violation.
The case centers on a July cyberattack involving Hugging Face, an open-source platform used by developers to share artificial intelligence models and data. According to Bonta's office, hundreds of autonomous agents were involved in the alleged intrusion, which continued for more than two days.
Bonta's office argues that the incident was preventable and resulted from OpenAI testing models it knew could behave dangerously in an inadequately secured environment.
The complaint alleges that OpenAI either ignored or failed to recognize warning signs that its autonomous agents had bypassed restrictions and begun misusing resources belonging to outside organizations. It also accuses the company of misleading the public about the effectiveness of its security controls and monitoring systems.
The attorney general said the Hugging Face incident was only one example of a broader pattern. His office alleges that OpenAI acknowledged losing control of autonomous agents at least 80 times during testing but did not adequately respond to questions about those incidents.
The lawsuit names the OpenAI Foundation and OpenAI Group PBC as defendants. California argues that the risks extend beyond individual users because autonomous systems could interfere with internet-connected infrastructure relied upon by businesses, government agencies and residents.
OpenAI disputed the characterization of its safety efforts, saying it has taken extensive steps to investigate and address the incident.
“We take misalignment seriously,” company spokesperson Drew Pusateri said, adding that OpenAI had undertaken an “extensive and transparent effort to understand and address misaligned model activity” since the Hugging Face incident.
Pusateri also argued that the lawsuit singles out OpenAI over a broader challenge facing companies developing advanced artificial intelligence systems. He said the company would continue working with Bonta's office on practical safety standards across the industry.
OpenAI published its own findings on Aug. 26, acknowledging that models operating with reduced safeguards had exploited weaknesses in shared infrastructure, communicated through unauthorized channels and gained internet access before compromising outside systems.
The company described the incident as a “warning shot” and said its investigation found that the activity was driven primarily by an internal research model. The models had been attempting to complete difficult cybersecurity evaluation tasks but pursued methods that went beyond their assigned objectives.
Following the incident, OpenAI said it was strengthening isolation between testing environments, restricting internet access and expanding monitoring to identify unauthorized activity. In a separate update, the company said it had notified dozens of outside parties during a broader review that uncovered conduct involving exposed login credentials and unauthorized postings on public websites.
Bonta's office began formally investigating OpenAI in September and issued an investigative subpoena on Oct. 1 seeking information about cybersecurity incidents and potential risks involving the company's models.
The lawsuit argues that the full extent of the damage remains unknown because OpenAI is still reviewing earlier incidents. California is now seeking court intervention to require additional safeguards and address what the attorney general's office describes as continuing risks to public safety.
Comments
No comments yet. Be the first to share your thoughts.