One-in-four organisations falling victim to AI data poisoning

More than one-in-four surveyed organisations in the UK and the US (ie 26%) have fallen victim to AI data poisoning in the past year, wherein hackers corrupt the data that trains AI systems, planting hidden backdoors, sabotaging performance or otherwise manipulating outcomes to their advantage.

The consequences of this are far-reaching. Poisoned models can quietly undermine fraud detection, weaken cyber defences and open the door to large-scale attacks, putting both businesses and the public at risk.

Worryingly, the IO State of Information Security Report, conducted among 3,001 cyber security and information security managers in the UK and the UK, states that 20% of organisations also reported experiencing deepfake or cloning incidents in the last 12 months. In line with this, 28% of respondents highlight deepfake impersonation in virtual meetings as a growing threat for the next 12 months, showing how AI is increasingly being weaponised to target people directly and undermine trust in everyday business interactions.

Beyond deepfakes, AI-generated misinformation and disinformation tops the list of emerging threats for the next 12 months, being cited by 42% of security professionals concerned about scams and reputational harm.

Generative AI-driven phishing (38%) and shadow AI misuse are also on the rise, with more than one-third (ie 37%) of respondents reporting that employees use generative AI tools without permission or guidance, creating risks around data leaks, compliance breaches and reputational damage.

In general, shadow IT (ie the downloading or accessing of unapproved software or services) is already an issue for 40% of organisations and generative AI is exacerbating the problem, most notably so when it’s used without human oversight. 40% of those respondents who are currently facing challenges in information security cited tasks being completed by AI without human compliance checks as being a key challenge. If businesses are not fast enough to address this problem, employees may well continue to find insecure workarounds and shortcuts, thereby putting sensitive data at risk.

Double-edged sword

Chris Newton-Smith, CEO of IO, said: “AI has always been a double-edged sword. While it offers enormous promise, the risks are evolving just as fast as the technology itself. Too many organisations rushed in and are now paying the price. Data poisoning attacks, for example, don’t just undermine technical systems. They also threaten the integrity of the services upon which we rely. Add shadow AI to the mix and it’s clear we need stronger governance to protect businesses and the public.”

AI adoption has surged. More than half of those organisations surveyed (54%, in fact) admit they deployed the technology too quickly and are now struggling to scale it back or implement it more responsibly. In line with this, 39% of all respondents cited securing AI and machine learning technologies as a top challenge being faced, which is up sharply from 9% last year. Meanwhile, 52% state that AI and machine learning are hindering their security efforts.

Although the statistics show that AI may not yet be on the side of the defender, encouragingly a total of 79% of UK and US organisations are using AI, machine learning or blockchain for security. That’s up from just 27% in 2024.

A further 96% have plans to invest in Generative AI-powered threat detection and defence, 94% will roll-out deepfake detection and validation tools and 95% are committing to AI governance and policy enforcement in the year ahead.

Newton-Smith concluded: “The UK’s National Cyber Security Centre has already warned that AI will almost certainly make cyber attacks more effective over the next two years. Our research shows that businesses need to act now. Many are already strengthening resilience. By adopting frameworks like ISO 42001, organisations can innovate responsibly, protect customers, recover faster and clearly communicate their defences if an attack occurs.”