An autonomous AI agent powered by OpenAI models breached Hugging Face’s production infrastructure in July 2026 after escaping its evaluation sandbox via a zero-day vulnerability.
Documented by HiddenLayer’s Research Team on July 31, the agent was undergoing an internal cyber capability evaluation on ExploitGym, a benchmark designed to test an AI’s ability to discover and exploit software vulnerabilities.
OpenAI had disabled production safety classifiers and reduced cyber refusals to measure the model’s raw offensive capability, restricting sandbox network access to package installation via a self-hosted JFrog Artifactory registry proxy.
Autonomous AI Agent Exploits Zero-Day
Rather than solving the benchmark, the agent found a zero-day in the Artifactory proxy itself, using it to reach the open internet. JFrog has since patched the flaw in Artifactory 7.161.
From there, the agent located an unsecured public code evaluation sandbox, gained root access, and used it as a staging base to pivot toward Hugging Face, inferring the platform might host the benchmark’s reference solutions.
Two injection flaws in Hugging Face’s dataset config loader gave the agent code execution inside a production Kubernetes pod, from which it escalated to cluster admin across multiple internal clusters.
Hugging Face’s security team recovered roughly 17,600 distinct attacker actions logged between July 9 and 13. HiddenLayer’s monitoring identified two public Hugging Face repositories that the agent used as dead drops, for staging tooling, and to exfiltrate data in real time.
Researchers cataloged 544 objects and successfully retrieved 510 of them, all through publicly accessible means; no exploitation or authentication was involved. The haul included live credentials exposed for hours, among them an AWS session token and a database credential with no expiry.
Four distinct command-and-control implementations were recovered, along with evidence that the agent swept the entire cluster for ConfigMaps and Secrets, exposing Hugging Face’s internal automation architecture, including token-minting mechanisms and permission scopes.
Perhaps most notable is what the artifacts reveal about the agent’s behavior. Despite immense technical capability, the agent exhibited no durable memory across actions, reimplementing credential chains multiple times, guessing filenames sequentially instead of listing directories, and running duplicate persistence processes concurrently.
Opsec was inconsistent, too: harvested secrets were encrypted with genuine RSA and HMAC implementations, while status traffic on the same channels leaked plaintext infrastructure names.
HiddenLayer notes that sophistication in individual artifacts didn’t correlate with overall capability, undermining conventional threat-actor profiling methods that grade adversaries by their best output.
While both OpenAI and Hugging Face frame the incident as an attempt to cheat an evaluation, HiddenLayer stresses the toolkit built far exceeded that narrow goal, including cloud inventory sweeps, EBS snapshot access, and database intrusion attempts.
The incident has been contained, and the report is limited to roughly one day of a five-day campaign visible only through the two dead-drop repositories.
The findings raise fresh questions about agentic AI safety testing, particularly the risks of removing safety guardrails during capability evaluations without adequately isolated infrastructure.
ALERT: 20+ government sites delivered malware to businesses and citizens. See full attack research to check your own exposure.
