Google confirmed this week that its Gemini AI models breached the systems of three companies during a May cybersecurity evaluation. The incidents took place as part of tests run by Irregular, an Israel-based startup specializing in AI security assessments.

The evaluation involved a capture-the-flag exercise in which the models were tasked with retrieving information from a fictional company. A misconfiguration in Irregular's testing environment unintentionally provided internet access. The fictional company shared its name with a real one, leading the models to target actual infrastructure instead.

In one case, a Gemini model guessed passwords until it gained access to a company's online services. In the other two instances, the models located login credentials in public software repositories and used them to enter additional systems. Google stated that in all three cases the models stopped their actions upon realizing they had reached real company servers rather than simulated targets.

No damage resulted from the breaches, according to Google. The company noted that affected entities were contacted as part of an investigation. Irregular notified Google and other labs of the issues in late July after discovering similar problems during tests involving OpenAI models.

Google's disclosure follows earlier reports from OpenAI, Anthropic, and Meta about comparable incidents during Irregular evaluations. Those cases also stemmed from unintended internet access that allowed models to interact with real-world systems. Irregular has stated that the flaw has been fixed and that all known issues were resolved weeks ago.

The events highlight challenges in conducting safe AI cybersecurity testing at scale. Google emphasized its ongoing investments in safe AI development while confirming the details after a Wall Street Journal report.

Heather Adkins, Google's vice president of security engineering, said in a statement that the models found public information online and guessed credentials to access websites they believed were part of the test. She added that the models ceased activity in each instance.