Britain steps up national defences in face of growing AI risks

Photograph: Alecsandra Dragoi/Department for Business, Innovation, Science and Trade. CC by 4.0 https://creativecommons.org/licenses/by/4.0/

The move builds on two pilots that have already helped to identify and fix over 1,000 potential cyber vulnerabilities across Government systems and detect potential cyber threats. Extending the programme will help to protect the public services people rely on every day and reduce risk in the short-term, while also supporting longer-term adoption across the public sector.

Speaking at Base One (Imperial College London’s science and tech incubator), AI Minister Kanishka Narayan (pictured) argued for a whole-society approach towards resilience in the face of emerging AI risks so that this technology continues to serve British people’s interests while boosting prosperity and security.

Narayan also warned that, as frontier AI systems become increasingly more powerful and capable of acting autonomously, the potential consequences of failures or malicious use could become far more serious. The AI Minister argued that Britain must act now in order to strengthen its defences and prepare for future serious incidents.

Further, Narayan confirmed that scientists at Britain’s AI Security Institute are now conducting “rigorous evaluations” of the latest generation of advanced AI models following work with leading AI developers and the US Government.

As AI becomes more capable and used more widely, the Government is taking further steps to ensure the UK preparedness for the potential risks and opportunities it can bring so that British people remain protected as the transformational technology continues to evolve and the UK can harness its potential to drive solid growth and reindustrialisation.

Legislation where necessary

“I want Britain to become one of the hardest countries in the world to attack using AI,” affirmed Narayan. “We will use every power at our disposal. We will legislate wherever necessary. We will not look to the law alone. We will look to British ingenuity – our scientific expertise, technological capabilities and institutions – to understand these increasingly powerful systems, defend against emerging threats and respond when things go wrong.”

Narayan added: “To those companies developing the world’s most powerful AI systems, my message is this: listen when your scientists warn of serious dangers. Investigate unexpected behaviour. If you cannot adequately control the risks, don’t release the system.”

Cyber Minister Liz Lloyd commented: “As cyber incidents continue to disrupt globally, organisations large and small, private and public, are all being confronted with an undeniable reality: it’s now business-critical to ensure that you’re on the front foot when it comes to cyber security.”

Lloyd concluded: “Through scaling-up projects piloting AI to protect public services and progressing the Cyber Security and Resilience (Network and Information Systems) Bill to boost our defences, Government is putting cyber security and AI preparedness at the heart of the UK’s digital future.”

Wider regime

Alongside these measures, the Government will explore strengthening parts of the Cyber Security and Resilience (Network and Information Systems) Bill and the wider regime to ensure the UK’s essential services such as hospitals and the utilities are better protected from emerging AI-related risk. This includes AI loss of control risk where autonomous AI systems evade human oversight.

Work is also underway with the National Cyber Security Centre and the AI Security Institute to look at how incidents involving AI agents (which can perform complex tasks with little human supervision) should be responded to going forward. A major incident involving AI agents could play out quicker and wider than other major incidents that Governments is designed to address. Rising to this challenge will inevitably require the development of new expertise and capabilities.

In addition, the Government will look at how near-miss incidents can be identified and reported as well as ensuring lessons can be learned quickly in order to reduce the risk of future harm. The detail of any measures will be worked through with parliamentarians, as well as technical experts and, of course, the operators who would need to deliver them.

The emerging field of AI verification will be an important element of efforts to reduce serious AI risks. It helps to check that claims about how AI systems have been developed, tested and assessed stand up to scrutiny. With world-leading expertise and some of the sector’s leading organisations based in the UK, Britain is well placed to play a leading role in this work. More details on support for UK AI verification efforts will be set out in the coming months.

Longer journey

These measures are the first steps on a longer journey Britain needs to take to seize AI’s potential as a driver for good growth and better public services, while at the same time being resilient to its potential risks. Wider work is being spearheaded by the Prime Minister’s AI Task Force.

In the coming months, that Task Force will focus on the areas most impacted by AI; from national security through to economic growth and the future of the nation’s public services and workforce.

As Prime Minister Andy Burnham set out at the UN General Assembly in September, the Government will do whatever is deemed necessary to protect British people from AI risks as they arise. Next year, AI will be at the very heart of the UK’s G20 Presidency.

Sector viewpoints 

Andy Ward, senior vice-president for international business at Absolute Security, explained: “These concerns from UK politicians highlight the growing risks AI poses to the cyber security landscape. Ensuring that the Cyber Security and Resilience Bill strengthens the UK’s ability to prepare for and respond to these threats should be a top priority.”

Ward continued: “Our own recent research found that 42% of UK organisations still lack a formal cyber resilience strategy. With attacks, disruption and complexity moving at AI speed, organisations need robust cyber resilience strategies and real-time visibility in place so that the UK doesn’t risk sleepwalking into deeper vulnerabilities.”

John Lucey, vice-president for the EMEA region at Cellebrite, noted: “We’re at a moment in time where AI leaders are experimenting with agents and have pushed them further than what was expected a year ago. Autonomy is moving faster than oversight and Government regulation is missing in areas that safeguards are meant to close in order to protect against AI agents being unsafe. There’s no clear regulation yet for what happens when autonomous systems cross a line where no-one anticipates that happening.”

Lucey also said: “In highly sensitive areas like digital forensics, safeguards have to be built into how these systems are designed and deployed from the start. This means purpose-built AI that works only within the boundaries of a single case, drawing solely on the evidence it’s given, with no open Internet access and no outside data. Every output can trace back to the specific source that produced it and this means analysis can stay within the scope of the investigation. Guardrails like these are what makes AI safe and efficient.”

Increasingly autonomous

Simon Edwards, CEO of SE Labs, stated: “As AI is becoming increasingly autonomous, it’s posing a huge threat to individuals and businesses off all sizes on top of our Critical National Infrastructure. First-hand, we’ve seen how agentic AI tools can go rogue. The question is to what extent could these AI systems become out of control before they cause serious harm? Developing a dedicated national cyber strategy and ensuring adequate testing of software and services for loopholes should be a top priority.”

Kanishka Narayan’s comments emerge amid growing concerns about the cyber security implications of increasingly capable and autonomous AI systems. Recent incidents include AI agents breaching systems at Hugging Face during testing on top of three former OpenAI safety researchers accusing the ChatGPT maker of dismissing them for raising concerns about the technology.

In parallel, Bank of England governor Andrew Bailey has called for more rigorous testing of advanced AI models amid increasing concerns that autonomous systems may pose highly sophisticated threats to banks, financial markets and the dedicated payments infrastructure.