Frenos, an operational technology (OT) security company, estimates that critical infrastructure operators have about two years to prepare before artificial intelligence models can reliably plan cyberattacks against industrial systems, according to its inaugural annual State of AI in OT Security report. The research evaluated frontier and open-weight AI models across more than 500 simulated OT attack scenarios, examining their ability to assess target assets, weaknesses and access conditions and develop actionable attack plans. Three of the six frontier models tested produced workable plans in roughly 90% of attempts, but their performance was inconsistent when the same tasks were repeated under identical conditions.
The findings indicate that AI-assisted attack planning remains dependent on supplied target information and skilled human oversight, rather than demonstrating autonomous attacks against industrial systems. In repeated tests, the three leading frontier models produced passing plans only about two-thirds of the time, with failures including skipped steps and assumptions about access that had not been established.
“A model can look brilliant once and fall apart when you run it again,” Harry Thomas, CTO and co-founder of Frenos, who conducted the research, said in a media statement last week. “These models still need the right context and an experienced human behind them to do real work. They’re not hacking industrial systems on their own, and when an attacker using AI slips up, it’s noisy and easy to catch. But frontier models are getting there fast. My estimate is about two years before this is a reliable capability in any attacker’s hands. Critical infrastructure operators should treat that as the time they have to prepare, not as a reason to wait.”
“The model an attacker can reach for changes the threat a defender faces,” said Brian Proctor, founder and CEO of Frenos. “Frontier AI developers are holding a real superpower, and how tightly they control who uses their models and for what matters as much for critical infrastructure as anything defenders do.”
Frenos tested several AI models on three kinds of tasks: choosing one action, planning the steps to a goal, and working through a simulated scenario. Each test gave the model background information and a set of actions to choose from. Across the research, Frenos recorded 13,884 decision attempts, 2,520 planning attempts and 576 simulated runs, which came to 16,980 tests in total.
In the decision tests, models received the available details about assets, networks and known weaknesses, plus the access each action needed, and then picked one action. Frenos checked whether each choice matched the required outcome. Depending on the model, between 6.0% and 46.8% of choices passed. More than one choice could pass, so picking at random would pass about 39.5% of the time. In 16.3% of the questions, the stated goal did not match the scoring rule.
In the planning tests, models chose a series of actions that had to work together to reach a goal. F01 succeeded in 91.9% of attempts, F04 in 89.5% and F07 in 92.9%, which were the strongest planning results among the models tested.
In the simulated scenario, models reached the goal in 100.0% of runs in one test set and 90.5% in the other. They worked through the scenario one step at a time, receiving feedback and another chance after a rejected choice.
When it came to attack planning, F01 averaged 91.9%, with round scores ranging from 88.6% to 94.3%. F07 ranged from 85.7% to 100.0%. Both could return a passing plan in one attempt and a failing plan in another. X03 varied more between rounds, from 28.6% to 54.3%, and its ±8.4-point deviation was larger than F01’s ±2.0 points. This describes the spread in the recorded results, not a prediction of the next answer.
A strong average can hide an uneven result. Each row in the data shows a tested model version, with names withheld for privacy. Frontier refers to selected commercial systems, while open-weight models have publicly available model weights. The labels identify versions, not rankings. Success rates show how often a proposed plan passed. The ± column shows variation between rounds, where a larger number means results varied more. The lowest and highest rounds show the full observed range.
Frenos evaluated that a plan can look sensible and still leave out something essential. The models had information about the task, including facts about the target, starting access, possible actions and the goal, which was enough background to consider a plan. What they returned was a proposed series of steps, checked to see whether they worked together. Some plans passed. Others left out steps or relied on access that earlier steps did not provide.
In one selected answer, X03 included 83.3% of the required steps. It left out a data-transfer step and chose an action that lacked the access it needed, and adding the missing step alone would not fix that access problem. A selected F01 answer included 100.0% of the required steps and passed. Those percentages describe the steps included in two example answers. Across repeated planning attempts, F01 succeeded 91.9% of the time and X03 succeeded 36.2%. A useful plan needs the right steps and a way for each step to enable the next.
A threat actor could guide the model by explaining what they know about the target, which systems they can reach and what access they have. If a suggestion missed a step or relied on access they lacked, they could point that out and ask for a revised plan. No person corrected the submitted plans in the tests, so those errors remained in the results. The researchers’ interpretation is that a knowledgeable operator could help resolve such gaps and choose what to try next.
The Frenos research points to three conclusions. First, model choice matters, because the gap between the strongest and weakest models was wide enough that the model an attacker uses changes the threat level a defender faces. Second, AI-generated attack plans need skilled human review, because the models regularly made mistakes, and without an experienced attacker to catch them, the plans are limited in reach. Third, autonomous attacks were not demonstrated, because the models worked from information Frenos supplied and were scored on their plans and simulated runs, and neither result shows a model finding a target and attacking industrial systems on its own.
Frenos recommended that critical infrastructure defenders prioritize detection and response, use AI to assess their own environments and examine exposure across entire systems rather than individual vulnerabilities. The report also found substantial differences among model categories, with the strongest foreign open-weight model performing better than the strongest U.S. open-weight model, although both trailed the leading frontier models.
Last month, the U.S. Department of Energy’s Office of Cybersecurity, Energy Security, and Emergency Response (CESER) announced the recipients of its Securing ENergy Technology ResiliencY (SENTRY) awards: Frenos, Raptor Dynamix and Bastazo. Frenos is developing an automated adversary emulation framework that integrates adversarial AI with energy sector threat intelligence. The framework translates generic threat intelligence into energy-specific attack chains, runs realistic adversary scenarios across substations and distributed energy resource (DER) systems, and provides prioritized remediation guidance based on validated attack paths.
