The U.K. government announced on Monday a package of measures to strengthen national defenses against growing risks from increasingly powerful artificial intelligence (AI) systems, including expanding use of advanced AI to identify and address cyber vulnerabilities, considering stronger legal protections for essential services, and preparing for serious incidents involving autonomous AI agents. The initiative builds on two pilot programs that have identified and helped fix more than 1,000 potential cyber vulnerabilities across government systems and detected potential cyber threats.
The government said the expanded program aims to reduce immediate risks to public services while supporting wider adoption of AI-based cyber defense across the public sector. These measures also include examining potential changes to the Cyber Security and Resilience Bill regime to better protect essential services, including hospitals and utilities, from emerging AI-related threats such as autonomous systems operating beyond human oversight.
The move builds on two pilots that have already helped identify and fix over a thousand potential cyber vulnerabilities across government systems, and detect potential cyber threats. Extending the programme will help protect the public services people rely on every day to reduce risk in the short term, and support longer-term adoption across the public sector.
The government is working with the National Cyber Security Centre (NCSC) and the AI Security Institute to develop approaches for responding to major incidents involving AI agents, which can perform complex tasks with limited human supervision and could cause incidents to unfold more quickly and across a wider area than conventional crises. Officials will also consider how to identify and report AI-related near misses, while the AI Security Institute evaluates the latest generation of advanced AI models.
Speaking at Imperial College London’s science and tech incubator, Base One, Kanishka Narayan, the country’s AI Minister, argued for a whole-society approach to resilience in the face of emerging AI risks, so that this technology continues to serve British people’s interests and boost our prosperity, as well as our security.
The Minister also warned that as frontier AI systems become increasingly powerful and capable of acting autonomously, the potential consequences of failures or malicious use could become far more serious. She added that Britain must act now to strengthen its defences and prepare for future serious incidents. The Minister also confirmed that scientists at Britain’s AI Security Institute are now conducting rigorous evaluations of the latest generation of advanced AI models, following work with leading AI developers and the U.S. government.
“I want Britain to become one of the hardest countries in the world to attack using artificial intelligence,” Narayan said. “We will use every power at our disposal. We will legislate wherever necessary. And we won’t look to law alone. We will look to British ingenuity: our scientific expertise, technological capabilities and institutions to understand these increasingly powerful systems, defend against emerging threats and respond when things go wrong.”
She added, “And to the companies developing the world’s most powerful AI systems, my message is: Listen when your scientists warn of serious dangers. Investigate unexpected behaviour. And if you cannot adequately control the risks, do not release the system.”
“As cyber incidents continue to disrupt globally, organisations large and small, private and public, are all being confronted with an undeniable reality – it’s now business-critical to ensure you’re on the front foot when it comes to cyber security,” Liz Lloyd, cyber minister, said. “Through scaling up projects piloting AI to protect public services and progressing a Cyber Bill to boost our defences, government is putting cyber security and AI preparedness at the heart of the U.K.’s digital future.”
Alongside these measures, the government will explore strengthening parts of the Cyber Security and Resilience Bill and the wider regime, to ensure the U.K.’s essential services such as hospitals and utilities are better protected from emerging AI-related risk. This includes AI loss of control risk, where autonomous AI systems evade human oversight.
Work is also underway with the National Cyber Security Centre and the AI Security Institute to look at how incidents involving AI agents – which can perform complex tasks with little human supervision – should be responded to. A major incident involving AI agents could play out quicker and wider than other major incidents that governments are designed to respond to, and rising to this challenge will require the development of new expertise and capabilities.
In addition, the government will look at how near-miss incidents can be identified and reported, as well as ensuring lessons can be learned quickly to reduce the risk of future harm. Details of any measures will be worked through with parliamentarians, as well as technical experts and the operators who would need to deliver them.
The emerging field of AI verification will also be an important part of efforts to reduce serious AI risks. It helps check that claims about how AI systems have been developed, tested and assessed stand up to scrutiny. With expertise and some of the sector’s leading organisations based in the U.K., Britain is well placed to play a leading role in this work. More details on support for U.K. AI verification efforts will be set out in the coming months.
Clearly, these measures are the first steps on a longer journey Britain needs to take to seize AI’s potential as a driver for good growth and better public services, while at the same time being resilient to its potential risks. Wider work is being spearheaded by the Prime Minister’s AI Taskforce. In the coming months, the Taskforce will focus on the areas most impacted by AI, from national security to economic growth and the future of our public services and workforce.
As the Prime Minister told the UN General Assembly in September, the government will do whatever is necessary to protect Britons from AI risks as they arise. Next year, AI will be central to the U.K.’s G20 presidency.
