AI-based agentic sources of security exploits aren’t new. The Open Worldwide Application Security Project (OWASP) published a paper that examines all kinds of agentic AI security issues with specific focus on model and application architecture and how multiple agents can collaborate and interact. It reviewed how users of various general-purpose agent frameworks such as LangChain, CrewAI and AutoGPT should better protect their infrastructure and data. Like many other OWASP projects, its focus is on how application development can incorporate better security earlier in the software lifecycle.
Andy Swan at Gray Swan AI led a team to publish an academic paper on AI agent security challenges. In March, they pitted 22 frontier AI agents in 44 realistic deployment scenarios that resulted in observing the effects of almost two million prompt injection attacks. Over 60,000 attacks were successful, “suggesting that additional defenses are needed against adversaries. This effort was used to create an agent red teaming benchmark and framework to evaluate high-impact attacks.” The results revealed deep and recurring failures: agents frequently violated explicit policies, failed to resist adversarial inputs, and performed high-risk actions across domains such as finance, healthcare, and customer support. “These attacks proved highly transferable and generalizable, affecting models regardless of size, capability, or defense strategies.”
Part of the challenge for assembling effective red team forays into your infrastructure is that the entire way incidents are discovered and mitigated is different when it comes to dealing with agentic AI. “From an incident management perspective, there are some common elements between agents and historical attacks in terms of examining what data needs to be protected,” Myles Suer of Dresner Advisory, an agentic AI researcher, tells CSO. “But gen AI stores data not in rows and columns but in chunks and may be harder to uncover.” Plus, time is of the essence: “The time between vulnerability and exploit is exponentially shortened thanks to agentic AI,” Bar-El Tayouri, the head of AI security at Mend.io, tells CSO.
