CyberArk unveils FuzzyAI for enhanced AI model security

CyberArk has announced a new open-source framework, FuzzyAI, designed to enhance AI model security by identifying vulnerabilities such as guardrail bypassing and harmful output generation.

FuzzyAI offers a comprehensive solution for AI model vulnerability testing, helping organisations address security challenges in both cloud-hosted and in-house AI applications.

It is available as open-source software on CyberArk Labs' GitHub page.

The use of AI models is transforming industries through improved customer interactions, automation, and internal process efficiencies. However, these benefits come with new security challenges that many organisations are not prepared to handle. FuzzyAI assists by providing a systematic approach to testing AI models against various adversarial inputs, thus identifying potential security weaknesses.

At the core of FuzzyAI is a powerful fuzzer that can expose vulnerabilities through more than ten distinct attack techniques, such as bypassing ethical filters and exposing hidden system prompts.

"It's imperative that when deploying AI, organisations learn from previous instances where new technologies have been introduced without proper consideration of the security consequences.

Relying on cybersecurity teams to play 'catch up' after AI security breaches, would be a costly and potentially devastating miscalculation," said Thomas Fikentscher, Area Vice President for ANZ at CyberArk.

Key features of FuzzyAI include comprehensive fuzzing techniques to reveal vulnerabilities like guardrail bypassing, information leakage, prompt injection, and harmful output generation.

The framework is extensible, allowing organisations and researchers to add their own attack methods, tailoring tests for specific vulnerabilities.

The tool also fosters community collaboration, promoting continuous advancements in adversarial techniques and defence mechanisms through a growing ecosystem.

Peretz Regev, Chief Product Officer at CyberArk, stated, "The launch of FuzzyAI underlines CyberArk's commitment to AI security and helps organisations take a significant step forward in addressing the security issues inherent in the evolving landscape of AI model usage.

Developed by CyberArk Labs, FuzzyAI has demonstrated the ability to jailbreak every tested AI model. FuzzyAI empowers organisations and researchers to identify weaknesses and actively fortify their AI systems against emerging threats."

CyberArk Labs will demonstrate the practical applications of FuzzyAI in a Capture The Flag (CTF) event at Black Hat Europe Arsenal. This event aims to showcase the real-world impacts of jailbreak vulnerabilities and emphasise the necessity for proactive mitigation strategies. Attendees of Black Hat Europe 2024 will have the opportunity to explore the tool's capabilities and applications.