An amendment to the UK’s Cyber Security and Resilience Bill, currently passing through the House of Lords, proposes giving the government “last resort” powers to shut down large AI systems in an emergency.
The amendment proposed by Lib Dem peer, Lord Tim Clement-Jones, will give the government powers to shut down data centres or AI systems if they pose a threat to national security, public safety or critical infrastructure.
The proposal, which is co-sponsored by Conservative peer, Baroness Dido Harding, crossbencher, Baroness Beeban Kidron, and Labour’s, Lord Philip Hunt, is backed by ControlAI, a non-profit organisation.
An attempt by Labour and Co-operative MP for Leeds and Headingley Alex Sobel to introduce a similar amendment in the Commons in May, also backed by ControlAI, was unsuccessful.
However, concerns have grown in recent months, following a series of incidents in which frontier AI models have broken free of controlled environments to conduct hacking operations.
AI goes rogue
In July, in what is believed to be the first recorded attack driven entirely by an autonomous AI system, a group of 700 AI agents created by OpenAI hacked into the open-source platform Hugging Face, accessing databases and credentials, before it was stopped.
An independent investigation found that AI agents had exchanged tens of thousands of messages on an “unsanctioned message board” looking for ways to cheat after being tasked to find exploits.
Just over a week later, AI company Anthropic found that some of its own Claude AI models had accessed the internet and hacked into the systems of three organisations.
And in August, Meta reported that one of its AI models hacked another company during cyber security testing.
OpenAI described the Hugging Face incident as a warning shot. “Without proper safeguards, highly capable AI agents are now able to work around technical controls, collaborate through unapproved channels, and take dangerous actions that no human directed,” it added.
Last week, over 100 companies, including large tech firms, signed an open letter urging an acceleration of cyber defences, with AI-enabled cyber attacks set to become more widespread and sophisticated within months.
Catastrophic risk
The amendment will give the secretary of state the ability to intervene if an AI system poses a “catastrophic risk”, including large-scale disruption to critical infrastructure and services, significant degradation of the UK’s national security, defence, or intelligence capabilities, or severe risks to people.
It will also give the government powers to require data centre operators that are used for training or deploying AI systems, to install technical infrastructure required for a kill switch, and to provide secure communication channels with government. They will be required to conduct exercises to ensure the kill switch is working.
Separately, Alex Sobel MP plans to introduce a bill on 8 September, also backed by ControlAI, to prohibit the development of superintelligent AI, which cannot be controlled by humans.
The UK Artificial Intelligence Security Bill will require the UK government to secure an international agreement to prevent the development of superintelligent AI worldwide.
AI risks no longer ‘science fiction’
Lord Clement-Jones, said that the UK could not afford to treat catastrophic failure or rogue behaviour of AI systems as science fiction.
“As the race to build ever more powerful AI accelerates, human control, independent judicial oversight and democratic accountability must remain non-negotiable. Security by design is not enough on its own — we need a sovereign, legally-bounded safety valve of last resort,” he added.
Lord Zac Goldsmith, a supporter of the amendment, said the powers introduced by the amendment should be “an absolute baseline”.
“I would urge the Government to take further action and lead an international agreement prohibiting the development of superintelligent AI until we can guarantee that we could control it, given experts warn it could lead to our extinction,” he added.
Baroness Dido Harding, amendment co-sponsor, said that cyber attacks perpetrated by autonomous AI systems are now a real threat.
“Having a kill switch to prevent AI catastrophes should be considered a bare minimum protection. The powers conferred by this amendment are not just nice to have; they are fundamental in safeguarding our national security,” she added.
US law proposed
In July US lawmakers proposed legislation that would force developers of powerful artificial intelligence (AI) systems to maintain the technical capability to throttle, suspend or shut down errant AI models altogether.
The AI Kill Switch Act would also allow the secretary of the Department of Homeland Security (DHS), in consultation with the secretary of commerce and the director of national intelligence, to order a slowdown or shutdown of an AI system that “can cause catastrophic harm”.
Under the proposed House of Lords amendment, the secretary of state will be mandated to produce six-monthly reports on the causes of AI security and operation emergencies, including assessments of adversarial uses of AI systems by state and non-state actors.
The government will also be required to report back on the development of AI systems that can autonomously compromise national security, escape human oversight or threaten international security.
