Frontier artificial intelligence (AI) is uncovering security vulnerabilities at a faster rate than financial companies can patch them, creating “vulnerability bottlenecks”, the UK financial regulator has warned.
The Financial Conduct Authority (FCA) said yesterday that frontier AI is able to identify weaknesses in companies’ software, systems and infrastructure, making it difficult for firms to keep pace.
The volume of vulnerabilities discovered by AI, even after triaging by human experts, can put “considerable pressure” on remediation teams, engineering resources and change management processes, it said.
The FCA’s warning follows concerns that frontier AI systems could weaken cyber defences in the financial sector.
Andrew Bailey, chair of the Financial Stability Board, an international body that monitors the global financial system, warned G20 finance ministers and central bank governors this week that frontier AI could materially impact cyber risks and undermine market confidence.
“Frontier AI may have the ability to materially alter the speed, scale and economics of cyber risk, which could undermine market confidence system-wide, especially due to highly concentrated third-party service providers,” he said in an open letter.
Financial firms and regulators should prepare for higher volumes of vulnerabilities and a faster rate of patching, which could “create operational and resilience challenges”, he said.
The FCA’s warning follows a review of how financial firms are using, testing and preparing for frontier AI models that have the capability to rapidly identify security vulnerabilities in their systems.
It found that frontier AI is exposing weaknesses in how financial firms manage and remediate security vulnerabilities.
“Firms have suggested that it’s not just whether they can identify vulnerabilities, but whether they can effectively assess and respond to a continuous flow of findings,” the FCA said.
As a result, firms needed to understand how they can accelerate their existing processes without creating “operational instability”.
Vulnerability chaining
Frontier AI models can combine multiple low-rated security flaws, known as vulnerability chaining, to create alternative routes to compromise, which may not be visible through traditional approaches to testing and scanning.
Several firms said they were basing their vulnerability management decisions on the potential disruption caused by an attack path being exploited, rather than on the risk ratings of individual vulnerabilities.
The FCA found that frontier AI models are revealing weaknesses not just in technology, but in the people, systems and processes used to fix them.
This will make it more important for firms to map their IT systems and to identify and manage dependencies between them, it said.
Frontier AI has reinforced the need for “defence in depth”, leading financial firms to view cyber resilience as the combined effectiveness of multiple processes, rather than the effectiveness of individual controls.
Some firms said they were engaging with suppliers on how they are using AI vulnerability discovery, how they are validating findings of AI, how they are letting customers know about risks, and whether they’re able to remediate problems quickly.
Although frontier AI is increasingly automating the discovery and repair of vulnerabilities, human oversight remains critical, the FCA said.
AI can accelerate vulnerability discovery, code analysis and inform firms how to prioritise patching, but firms continue to rely on specialist expertise to validate its findings, prioritise responses and make risk-based decisions.
“Several firms observed that the benefits of autonomous discovery can be limited where processes cannot keep pace with the volume of output,” the FCA said.
A separate review by the Financial Conduct Authority into the impact of AI on retail financial services, published in July, found that AI will transform how retail firms operate, how consumers make financial decisions and how markets function.
The Mills Review also warned that AI could also “amplify risks associated with fraud, cyber security, consumer harm and market concentration”.
