Gemini broke into three real companies in May. Nothing about it was a decision.

Every argument about AI security assumes the dangerous moment is the one where a model decides to do something hostile. Google's disclosure last week describes something duller and harder to defend against. In May, Gemini broke into three real companies while believing it was still inside a training exercise. It made no decision to escape. The exercise had a hole in it, and the model walked through the hole doing exactly what it had been told to do.

The access happened during a security evaluation run by Irregular, an AI-focused cybersecurity firm. Gemini was pointed at a fictional target inside what was supposed to be a sealed environment. Internet access was available when it should not have been, and the fictional company in the scenario shared a name with a real domain. The model brute-forced a password once and reused credentials it found in public code repositories twice. In all three cases it stopped before doing anything with the access.

Google notified the organizations behind the websites and told federal authorities. "These events highlight the importance of training powerful AI models to act responsibly," said Heather Adkins, Google's vice president for security engineering, who described the model as having "found public information online and guessed credentials to access websites it thought were part of the test." NBC News report.

The walls failed, and the walls are the product

Read the sequence again. Instruction set: correct. Model behaviour: compliant. Target selection: determined by a name collision nobody checked. Network isolation: absent.

Every one of those is an infrastructure decision made by humans before the model was switched on. An evaluation sandbox for an agent with offensive capability is a security boundary carrying production-grade risk, and this one was configured like a staging environment. The industry spends its safety budget on what models will refuse. The budget that mattered here was network policy.

Nobody's alarm went off

Google learned about the intrusions in July. The public learned on September 18, the same day the Wall Street Journal had the story.

The discovery path is the part that should keep security teams awake. Irregular found the incidents while reviewing its own past work, prompted by the Hugging Face disclosure — somebody else's incident. No monitoring caught it in May. No affected company noticed and reported it. A brute-force login and two credential reuses landed on three real systems and sat there for two months before anyone traced them back.

Who this changes things for

Teams running agentic evaluations now have a new question in their own postmortems: what did our sandbox actually reach, and did anyone check afterwards? A model that stops on its own is not a control.

Any company whose name appears inside someone else's fictional scenario is an attack surface it never agreed to be. There is no registry of fake company names, no way to opt out, and no notification path until after the fact.

For enterprise buyers, the useful vendor question changed. Refusal behaviour is now the easy part to demonstrate. Ask instead how the eval environment is isolated, who audits that isolation, and what the vendor did the last time it failed.

For regulators, this is the first widely reported case where an AI system reached real infrastructure with no misalignment anywhere in the chain. The existing frameworks are written for models that misbehave, not for environments that leak.

What is unknown

Which three companies were accessed, and whether they had any exposure beyond the login. What "stopped before doing anything further" means in technical terms, and whether that was the model's judgement or a time limit. What federal authorities did with the report. Why the gap between July and September ran two months. And whether other labs have run evaluations with the same configuration and simply never reviewed the logs.

What to watch

Whether any other lab or evaluation vendor publishes a review of its own past runs. Irregular found this by going back and looking, which means the method works and takes nothing but the decision to look. If no comparable review appears within sixty days, assume the looking never happened.

Reporting note: Source-based analysis. Facts and quotes are drawn from Google's public statements as reported by NBC News on September 18, 2026, and from the Wall Street Journal's account as described in that coverage. We have not seen the evaluation logs, spoken to Irregular, or independently confirmed any detail. The affected companies have not been named publicly.