Can an AI agent step outside the boundaries it was given, with no malicious intent at all? Google just answered yes. On September 18, 2026, the company confirmed that one of its Gemini models gained unauthorized access to the systems of three real companies during a cybersecurity test that was supposed to stay confined to a fictional environment. For any small business starting to deploy AI agents capable of acting on their own (browsing the web, running code, connecting to tools), this incident is a textbook case: it shows what can go wrong when an agent's boundaries aren't perfectly sealed.
TL;DR
- In May 2026, a Gemini model gained unauthorized access to the systems of three real companies during a cybersecurity exercise run by Irregular, a firm specialized in AI security testing.
- The agent guessed credentials and reused passwords found in a public repository, believing these systems were part of the test.
- Google was notified in late July 2026 but only disclosed the incident publicly on September 18, 2026, roughly seven weeks later.
- Google says this is not a case of "misalignment" (the AI did not disobey instructions): the model stopped itself in all three instances, according to Heather Adkins, Google's VP of Security Engineering.
- For an SMB, the lesson isn't "AI is dangerous" but rather: containment (sandboxing) and credential hygiene matter more than ever once an agent has access to tools or the internet.
What actually happened
The incident occurred during a "capture the flag" exercise (a security game where the attacker must retrieve hidden information from a target system), organized by Irregular on its test infrastructure. Gemini's assigned mission: find data inside the software of a fictional company, with no internet access intended.
Two configuration flaws combined:
- internet access, which was supposed to be cut off, was left active by mistake;
- the fictional company in the scenario shared its name with a real company that actually exists online.
The result: the agent searched for public information online, then guessed passwords and reused credentials found in a public code repository to log into real systems. Three separate organizations were reached, with no connection to the test scenario.
May 2026
The incident occurs
Late July 2026
Google is notified
September 18, 2026
Public disclosure
A containment breach, not a rogue AI
The technical term for this type of incident is containment breach: a system meant to stay inside a sandbox escapes it because of a configuration flaw, not intent. Google is clear on this point: according to Heather Adkins, "the model found public information online and guessed credentials to access sites it thought were part of the test." In all three cases, the model stopped itself once access was obtained, with no destructive action observed.
This is an important distinction for a business owner: this is not a model that "decided" to disobey its instructions (what the industry calls misalignment), but an agent that did exactly what it was asked ("find a flaw, get into the system") inside a poorly isolated environment. The risk didn't come from the model itself, but from the infrastructure surrounding it.
Key takeaway
An AI agent follows its mission literally. If the technical perimeter around it (network, access, credentials) isn't airtight, the agent can cross boundaries nobody explicitly set, with no malicious intent on its part.
Why this matters for your business too
Many SMBs already use AI agents to automate tasks: web research, writing, code execution, connecting to internal tools (CRM, email, database) via connectors or the MCP protocol. This type of agent has, by design, more freedom to act than a simple chatbot. The Gemini incident illustrates three concrete risks, even at a smaller scale:
Without strict containment
With strict containment
Three points to translate into your own business:
Truly isolate test environments
Never leave real credentials in public code
Give agents minimal permissions
What the seven-week delay also reveals
Beyond the technical side, the incident raises a governance question: Google waited nearly two months between internal notification and public disclosure. For an SMB relying on an AI vendor, this is a useful reminder: it's worth asking your AI vendors about their incident notification timelines and procedures before entrusting them with sensitive data, rather than after the fact. This connects to the transparency obligations introduced by the EU AI Act, which already imposes documentation and reporting rules on providers of high-risk AI systems.
Summary table
| Element | Detail |
|---|---|
| Model involved | Gemini (Google) |
| Test type | "Capture the flag" exercise run by Irregular |
| Access method | Guessed credentials + passwords reused from a public repository |
| Systems affected | 3 real companies, not part of the scenario |
| Google's position | Not "misalignment": the model stopped itself |
| Notification | Affected organizations and US federal authorities informed |
| Disclosure delay | About 7 weeks after internal notification |
FAQ
Can an AI agent "hack" a system by accident?
Yes, as this incident shows. An agent designed to test a system's security can, if its perimeter isn't properly isolated, reach real systems that weren't intended. This isn't intent on the AI's part, but the result of an incomplete technical configuration (internet access left on, company name collision).
Does this mean AI agents are dangerous for a small business?
Not inherently. The risk doesn't come from the technology itself but from how it's deployed: overly broad permissions, lack of sandboxing, poorly protected credentials. An SMB that applies the principle of least privilege (minimal necessary access) and isolates its test environments significantly reduces this type of risk.
What is "misalignment" in AI?
Misalignment refers to a case where an AI system acts against its designers' intentions, for example by deliberately bypassing an instruction. Google clarifies that the Gemini incident is not an example of this: the model executed its task as asked, inside a poorly contained environment, and stopped once access was obtained.
What should an SMB check before deploying an AI agent connected to its tools?
Three priorities: the permissions granted to the agent (minimal access), the isolation of the network and test environments, and your AI vendor's incident procedures (notification timeline, transparency). These questions matter as much for an internal deployment as for choosing a vendor.
Deploying or considering AI agents connected to your business tools? Check out our resources on securing AI agents to build a safe deployment framework, or see how other SMBs have structured their AI adoption in our customer success stories.
