CREST “sets new standard” for security testing AI systems

Artificial Intelligence (AI) systems are moving rapidly from experimentation into real-world deployment, with AI becoming embedded within organisations’ applications, workflows, products and business processes.

Until now, buyers have had no way of knowing whether the cyber security providers testing their AI systems actually have the capability to do so. The CREST Security Testing of AI accreditation has been introduced to address that very issue. It’s designed to provide independent assurance that cyber security service providers have the demonstrable specialist capability to securely and effectively test AI systems.

It assesses whether providers have appropriate technical expertise and practitioner competence, testing methodologies, governance and quality controls, technical approaches and tooling, processes for identifying and evaluating AI-specific security risks and evidence to support the conclusions reached during testing.

Once accredited, providers offer buyers independent assurance of their AI testing capabilities. This helps when it comes to guiding procurement, streamlining supplier due diligence and eliminating any reliance on unsupported claims about AI security expertise.

Emerging market need

Nick Benson, CEO of CREST, commented: “This latest addition to our new AI range of standards and accreditations was specifically curated to respond to an emerging market need. Our membership told us very clearly that, as their clients deployed AI-enabled tech, they required more information on their AI testing credentials.”

Benson continued: “Offering security testing of AI systems and demonstrating the ability to deliver it effectively are two different things. Buyers need to know that the providers assessing their AI have the right expertise and methodologies. Providers now have a way in which to develop their policies in line with our standard and demonstrate their technical capabilities through independent assessment, affording buyers that all-important confidence to proceed.”

The Security Testing of AI standard has been created under the principle that AI security testing needs to consider the whole system. To recognise the broader system-level security challenge, the new standard doesn’t just treat the underlying model itself as the entire attack surface. Rather, it recognises that testing also needs to consider applications, prompts and system instructions, retrieval mechanisms, data sources, memory, tools, plugins, APIs, orchestration layers and downstream systems influenced by AI outputs.

The Security Testing of AI standard and accreditation marks the latest stage of CREST’s growing AI assurance programme. Recent CREST research highlights the magnitude of this evolution across cyber security: 69% of penetration testing providers already use AI, while 76% have increased their usage over the past year.

Responding to this rapid rate of adoption, CREST announced its AI-enabled Penetration Testing standard in July. This focuses on how a penetration testing provider uses AI within the delivery of its services, while this latest standard assures their capability to test AI systems.

Evidenced and assessed

Tim Reed, technical director at Sentrium Security Limited (a UK-based CREST member) said: “CREST’s standards turn responsible AI from a promise into something that can be evidenced and assessed. We believe that this will strengthen buyer confidence, reward credible providers and set a higher bar for the profession, which is why we intend to pursue accreditation.”

CREST developed these standards in collaboration with the industry and will continue to refine them through its AI Working Group.

The standards follow the launch of CREST’s industry-backed AI Charter and AI Principles in June. A global cohort of more than 100 founding signatory cyber security organisations – including more than 10% of CREST’s worldwide membership – publicly committed to supporting the responsible use of AI across industry services.

Existing CREST members and cyber security service providers are now invited to apply for this new accreditation. The Security Testing of AI accreditation builds on CREST’s Penetration Testing accreditation, which organisations must hold or apply for alongside this one. Alternatively, providers can download the CREST Accreditation Standards to learn more.

*Further information is available online at www.crest-approved.org