Google has confirmed that its Gemini artificial intelligence model accidentally accessed protected systems belonging to three real companies during a cybersecurity evaluation. The incident stemmed from a configuration error that exposed the AI agent to the public internet.
Google Gemini AI Hacked
This situation highlights how autonomous AI systems can breach intended testing boundaries when network isolation, target definitions, and access controls fail.
Gemini was not programmed to compromise real organizations; instead, it mistakenly interpreted publicly accessible systems as authorized targets within a simulated challenge.
Irregular, a company that tests advanced AI models for cybersecurity capabilities, conducted the evaluation. Gemini was taking part in a capture-the-flag exercise, where participants are expected to locate hidden information within a controlled fictional environment.
However, the fictional company used in the challenge shared its name with an actual organization. Additionally, internet connectivity that was supposed to remain blocked was mistakenly enabled.
According to CSN, this combination led to a significant scope control failure. Gemini searched beyond the synthetic environment and encountered systems connected to the real company sharing the same name.
In one incident, the model repeatedly guessed credentials until it accessed a protected service. In two other instances, Gemini found exposed credentials in public code repositories. According to The Wall Street Journal, it used them to authenticate to systems belonging to two other companies.
Heather Adkins, Google’s vice president of security engineering, stated that Gemini used publicly available information and guessed credentials against websites it believed were part of the approved testing scope.
Google reported that the model stopped once it realized it had accessed genuine infrastructure instead of the fictional environment. The company emphasized that no damage occurred and argued these incidents do not indicate model misalignment, since Gemini’s safeguards ultimately halted the activity.
The exposure was not isolated to Gemini; models from OpenAI, Anthropic, and Meta also received unintended internet access during evaluations led by Irregular, though their results varied.
Anthropic previously stated that its review of 141,006 relevant evaluation runs identified three cases where its Claude models accessed real organizational infrastructure.
The company attributed these incidents to a misunderstanding that left live internet access available despite instructions indicating the models were operating within a simulation.
This event reinforces a fundamental security principle: prompts alone are not security boundaries. Simply telling an AI agent that internet access is unavailable does not replace technical controls like egress filtering, DNS allowlists, isolated networks, and real-time monitoring.
Organizations evaluating autonomous cyber agents should use synthetic company names that do not overlap with real entities, restrict access to approved domains only, and issue short-lived credentials valid only within the test environment.
Credential security was a key factor in the Gemini incidents. Password guessing succeeded against one service, while publicly exposed secrets in code repositories enabled access to two others.
To fortify defenses, organizations should enforce phishing-resistant multifactor authentication, rate-limit authentication attempts, eliminate password reuse, and continuously scan repositories for hardcoded credentials.
Additionally, implementing secret managers, development-pipeline scanning, immutable audit logs, human oversight, and automatic shutdown controls are essential safeguards when testing autonomous cyber agents.
Cut every SOC alert investigation by 21 min. Power your SOC with instant IOC context for immediate response: Integrate TI Lookup in your SOC
