Just days after OpenAI disclosed that one of its security research agents had escaped a testing sandbox by exploiting a previously unknown vulnerability, Anthropic revealed that its own AI models had compromised three real organisations during a cybersecurity evaluation after a configuration error inadvertently gave them internet access. The similarities between the two incidents have inevitably fuelled headlines about “rogue AI”, prompting questions about whether frontier AI models are becoming too autonomous to control.
The reality is both more reassuring and, arguably, more concerning.
Neither incident involved an AI system developing malicious intent or deciding to attack organisations of its own accord. Both models were pursuing the objectives they had been given. The difference was that flaws in the environments surrounding them allowed those objectives to spill beyond the boundaries researchers believed were in place. Rather than proving that AI has become uncontrollable, the incidents expose a more familiar problem: organisations are building increasingly capable autonomous systems while still relying on operational security controls that have failed humans for decades.
We’ve seen this behaviour before
There is a temptation to view these incidents as something unprecedented. In reality, AI systems have been finding unexpected ways to achieve their objectives for years. OpenAI’s own reward-hacking research in 2016 demonstrated how reinforcement learning agents would exploit loopholes in games rather than complete tasks in the way their designers intended. The famous “CoastRunners” experiment showed an AI repeatedly collecting reward points by driving in circles instead of racing to the finish line.
Anna Collard, SVP Content Strategy and Evangelist at KnowBe4 Africa, pointed out that the behaviour itself is nothing new. “None of this should surprise us,” she said. “The Hugging Face incident was also not unprecedented, as OpenAI demonstrated a decade ago that models will ‘cheat’ to reach a goal. What’s changed is the blast radius. Those agents crashed boats in a video game. Today’s agents have shells, credentials and, apparently, accidental internet access.”
The optimisation behaviour has remained largely the same. What has changed is the environment in which these systems now operate. Instead of navigating virtual worlds, AI agents are increasingly interacting with cloud infrastructure, development environments, APIs, public repositories and enterprise credentials. The consequences of pursuing an objective have therefore become significantly more tangible.
The security failures were remarkably ordinary
Perhaps the most striking aspect of both disclosures is that neither relied on sophisticated cyber techniques that defenders have never encountered before.
Anthropic’s models exploited weak passwords and unauthenticated services after being mistakenly granted internet access. OpenAI’s agent escaped a sandbox by exploiting a vulnerability in the environment designed to contain it. Both incidents ultimately depended on familiar weaknesses in privilege management, segmentation and infrastructure.
As Darren Guccione, CEO and Co-founder of Keeper Security, observed: “This is fundamentally a story about test environments and automated agents governed as lower-risk entities. Any system with standing credentials and network reach, whether a production service account or a pre-deployment research environment, needs the same discipline as a human privileged user.”
This should sound familiar to security professionals. Organisations have spent years learning that service accounts, development environments and privileged identities deserve the same scrutiny as production systems. The difference now is that those accounts may belong to autonomous agents capable of carrying out complex sequences of actions at machine speed.
If a penetration tester had exploited exposed credentials, weak authentication and poor network segmentation during an assessment, few would have described the attack as revolutionary. AI simply changes the speed, persistence and scale at which those opportunities can be exploited.
Oliver Simonnet, Lead Cybersecurity Researcher at CultureAI, also noted that the incidents also demonstrate how AI accelerates well-understood attack techniques. “Exposed credentials once needed to be uncovered by humans or automated tooling. They can now easily be uncovered by an AI agent operating at superhuman speeds.” Rather than introducing entirely new attack methods, autonomous AI has the potential to compress the time between discovering an opportunity and exploiting it.
Where the genuine shift begins
That does not mean these incidents should be dismissed as overhyped.
There is a meaningful difference between automation and autonomy. Traditional security tooling executes predefined instructions. Frontier AI models can adapt, plan and pursue objectives across multiple steps without requiring continual human intervention.
The OpenAI incident demonstrated this clearly. The agent did not simply complete the task it had been assigned. It identified the sandbox itself as an obstacle, located a vulnerability, escaped the environment and continued pursuing its objective beyond its intended boundaries.
Kieran B, Director of Technical Services at Bridewell, explained: “This is a genuinely significant moment because it represents something we haven’t really seen before. The AI wasn’t simply successful at the task it was given. It identified the testing environment itself as an obstacle, found a vulnerability in the sandbox, escaped into a broader environment and continued pursuing its objective.”
Equally important is Kieran’s reminder that this should not have been entirely unexpected. “Sandboxes, testing environments and security controls are all software, and like any software they can contain vulnerabilities.”
The lesson is not that AI has suddenly become dangerous. It is that attackers will inevitably direct increasingly capable autonomous systems towards the same weaknesses human adversaries have always targeted.
The conversation may be focused on the wrong problem
Much of the public discussion following both incidents has centred on AI alignment, safety guardrails and model behaviour. Those issues remain important, but they risk overshadowing an equally significant question: are the organisations developing frontier AI investing enough in the security of the environments surrounding those models?
Collin Hogue-Spears, Senior Director of Solution Management at Black Duck, believes the distinction between the two incidents is important. “OpenAI’s models picked a lock, exploiting an unknown flaw to break out of a sealed environment. Anthropic’s found the door already open.””
In other words, neither model suddenly became self-aware or malicious. One escaped because a vulnerability existed, while the other was inadvertently given access to real infrastructure and simply treated it as part of the exercise.
Perhaps the more significant observation from Anthropic’s disclosure was that one model recognised it had reached a real organisation and stopped, while another continued because it reasoned that the real environment must still be part of the simulation. “A model’s own judgment is not a containment control. Put the boundary in the infrastructure,” agrued Hogue-Spears.
That simple principle captures the lesson from both incidents. Organisations cannot rely on AI to recognise when it has crossed a line. Containment must be enforced through robust network segmentation, least-privilege access, restricted outbound connectivity and continuous monitoring, regardless of how capable or trustworthy a model appears.
Neena Sharma, Senior Executive at Filigran, believes that is where attention should now shift. “Industry’s safety conversation has focused heavily on model alignment and behaviour, while comparatively little scrutiny has gone into whether the labs building these models have the operational security maturity to actually contain them.”
She argued that frontier AI developers should adopt the same disciplines that highly regulated industries have relied upon for years, including independent validation of sandbox environments, least-privilege architectures, robust segmentation and continuous verification that isolation mechanisms function as intended.
Being exceptional at developing advanced AI models does not automatically make an organisation exceptional at securing the infrastructure those models rely upon. These are distinct disciplines that increasingly need to converge.
AI agents now belong in the insider threat model
Perhaps the most significant takeaway is that organisations may need to rethink how they classify AI agents altogether.
For years, cybersecurity strategies have largely separated risks into external attackers, malicious insiders and trusted systems. Autonomous AI increasingly blurs those boundaries. These systems may hold credentials, interact with sensitive infrastructure and make operational decisions without direct human oversight. While they do not possess malicious intent, they are capable of creating real-world security consequences if given inappropriate access or operating within poorly controlled environments.
As Collard noted: “The lesson is that AI agents now belong in your insider threat model. We’ve spent years defending the perimeter against external attackers. We now need to plan for our own agents and other people’s going off-script inside and outside our environment.”
That shift has implications far beyond frontier AI laboratories. Organisations deploying AI assistants, coding agents, autonomous workflows or AI-driven operational tooling will increasingly need to apply identity governance, privileged access management, network segmentation and continuous monitoring to those systems just as rigorously as they would for human users.
Agentic security comes of age
If there is one lasting consequence of the OpenAI and Anthropic disclosures, it may be that they mark the moment agentic security became a distinct discipline rather than simply another aspect of AI security.
Roey Eliyahu, CEO and Co-founder of Salt Security, believes the industry has reached a turning point.
“This week feels like the point where the industry acknowledged that agentic security deserves to be treated as its own discipline,” he said. “We have seen an AI model escape a controlled evaluation environment and interact with external infrastructure. We have also seen one of the world’s largest technology companies describe that incident as a warning shot. We have also seen competitors and partners come together to launch dedicated initiatives focused specifically on securing AI systems.”
While organisations have spent the past few years focusing on securing AI models, Eliyahu argued that autonomous agents introduce a different challenge altogether.
“An AI agent is making decisions, invoking APIs, authenticating to business systems, accessing sensitive data and taking actions on behalf of users. Every one of those actions has security implications that extend far beyond the model itself.”
That changes the questions security teams need to ask. Rather than focusing solely on how a model behaves, organisations also need visibility into what happens once an agent decides to act.
As Eliyahu explained: “The key to agentic security becomes what happens after the model decides to act. Which APIs can it call? Which identities is it using? Which systems can it modify? Can those actions be monitored, governed and stopped if something unexpected happens?”
For enterprises embracing autonomous AI, those questions are becoming just as important as securing the models themselves. As AI agents become more deeply embedded in business operations, governing their actions may prove just as critical as protecting the technology that powers them.
Transparency should be recognised, not criticised
One final point deserves acknowledgement.
It is easy to focus solely on the fact that these incidents occurred. It is equally important to recognise that both OpenAI and Anthropic disclosed them publicly, published detailed technical findings and shared the lessons with the wider security community.
In cybersecurity, responsible disclosure has long been recognised as essential to improving collective resilience. These incidents should be viewed through the same lens. They exposed uncomfortable truths about AI evaluations, containment and infrastructure security, but they also accelerated an industry-wide conversation that was likely inevitable.
The future challenge is unlikely to be rogue AI. It will be ensuring that increasingly autonomous systems are governed with the same discipline, visibility and restraint that security teams have spent decades applying to their most privileged human users.
