Over the past several months, Mozilla ran an agentic harness powered by Claude Mythos Preview across Firefox’s source code, identifying 271 security bugs that were fixed in Firefox 150, with additional fixes shipped in versions 149.0.2 and 150.0.1. Over 100 people contributed code to get those patches out.
The bugs spanned a wide range of subsystems. Among the disclosed reports: a 15-year-old flaw in the HTML
Building the harness
Mozilla engineers began with small-scale experiments using Claude Opus 4.6, targeting sandbox escapes specifically. Brian Grinstead, a Mozilla Distinguished Engineer, described the core requirement for making the system work at scale.
“This pipeline is extremely reliable at filtering out false positives, so long as you have a clearly defined success condition to validate against. Memory corruption issues are especially easy to validate: either you trigger Address Sanitizer or you don’t (in which case you tell the agent to keep working until it does),” Grinstead said.
False positives did occur in limited numbers. “We did see a small handful of false positives, primarily caused by changing some precondition in order to trigger an issue that would otherwise be valid (e.g., enabling a testing preference or using a private API). We’ve seen fewer than 15 of these total, and when we see them we update the harness to prevent similar issues in the future,” he added.
Once working, the team parallelized jobs across multiple ephemeral virtual machines, each assigned a specific target file. Findings were written to a bucket, then fed into a security bug lifecycle covering deduplication, triage, classification, and patch management.
Containment of modified builds
Crafting sandbox escape proofs required the model to patch Firefox source code, a step that raised questions about containment. Grinstead explained the controls in place.
“Any source code changes made to craft sandbox escapes are only used to generate bug reports. They never land in the upstream Firefox source code, nor are they published anywhere outside of the bug tracker. Scanning happens entirely within isolated VMs that have a local copy of Firefox’s open source codebase, with no means to publish their changes. After completing the analysis, any findings are written into an internal database and the VM is destroyed,” he said.
What the model could not break
Equally notable is what the system failed to exploit. Mozilla engineers observed the harness repeatedly attempting to escape the process sandbox via prototype pollution in the privileged parent process, a technique that had succeeded for external researchers in prior years. An architectural change to freeze those prototypes by default blocked every attempt.
The team plans to integrate the analysis into continuous integration to scan patches as they land, shifting from file-based to patch-based scanning.
Vendor accountability and defensive pressure
“What stood out to me in the report was how Mozilla’s past architectural decisions to apply layered defences within Firefox resulted in Mythos repeatedly failing to exploit vulnerabilities. This reinforces what many cybersecurity professionals have advocated for years that cybersecurity needs to be included in all software at the design, development, and the deployment stages,” Brian Honan, a cybersecurity and data protection expert and owner of BH Consulting, told Help Net Security.
On vendor accountability, Honan pointed to procurement as the primary lever available to security leaders. “The core leverage CISOs have with vendors who are not doing this kind of hardening is their budget. Working closely with their company’s procurement functions CISOs should look to include security requirements as mandatory elements of purchasing decision making process. Part of those requirements should include asking vendors during a tender or renewal process what secure practises they have around the development of their product or service and whether they are incorporating AI-assisted code analysis into their secure development lifecycle,” he said.
Mozilla’s recommendation to other development teams is to begin with simple prompts and iterate, treating the harness as infrastructure that compounds value with each model generation upgrade.
Download: Secure Foundations for AI Workloads on AWS
