An AI model may be designed to behave safely, but that doesn’t mean the system delivering it is secure, according to new research from the University of Maryland.
The study—currently under review for presentation at a major conference on neural processing systems later this year—found that critical security vulnerabilities often lie not within AI models themselves, but in the complex software infrastructure that serves them to millions of users. To uncover these flaws, researchers developed a testing tool called GRIEF (Greybox Inference Engine Fuzzer), which uses a cybersecurity technique known as fuzzing—automatically generating large numbers of varied, overlapping requests—to expose weaknesses that conventional testing can miss.
The findings challenge a common assumption that AI safety depends primarily on monitoring a model’s inputs and outputs.
“Many people assume that if an AI model doesn’t hallucinate or get tricked by unsafe prompts, the system is secure,” said Michelle Mazurek, a professor of computer science with an appointment in the University of Maryland Institute for Advanced Computer Studies (UMIACS) and director of the Maryland Cybersecurity Center (MC2). “Our research shows that the software infrastructure managing and scheduling those requests introduces a completely separate, highly complex layer of vulnerability.”
The study was led by Yunze Zhao, a second-year doctoral student in computer science advised by Mazurek. The research team also includes Alan Liu, assistant professor of computer science with an appointment in UMIACS and a core member of MC2, and Yibo Zhao, a second-year doctoral student in computer science. They collaborated with Yuchen Zhang, an assistant professor at Beijing Normal–Hong Kong Baptist University, who was a postdoctoral researcher at New York University when the study was conducted.
To run large language models (LLMs) efficiently at scale, companies rely on inference engines—the software that manages AI requests and generates responses for thousands of users simultaneously. To maximize performance, these systems use techniques such as dynamic request batching, memory caching and multi-user scheduling.
Because inference engines process many users' requests at the same time using shared computing resources, subtle software bugs can emerge that conventional security testing often misses. Traditional tools typically evaluate one request at a time, overlooking vulnerabilities that appear only when multiple users interact with the system simultaneously.
GRIEF addresses that gap by generating synchronized, overlapping workloads that simulate real-world traffic, allowing researchers to observe how AI serving systems behave under realistic operating conditions.
“These failures only appear when multiple valid requests interact concurrently,” Yunze Zhao said. “By bombarding the engines with synchronized, complex traffic, we can test whether these systems truly isolate users, preserve data integrity and remain online under heavy loads.”
When researchers tested GRIEF on vLLM and SGLang—two of the world's most widely used open-source LLM serving systems—it uncovered 16 vulnerabilities. Developers confirmed 10 of them, and four were assigned Common Vulnerabilities and Exposures (CVE) identifiers, the standard system for tracking significant cybersecurity flaws.
The vulnerabilities crossed several critical security boundaries. In some cases, one user's private prompt appeared in another user's session. In others, the system silently corrupted responses to unrelated queries or froze entirely under specific traffic conditions, causing a denial-of-service failure.
The researchers also found that these infrastructure failures can easily be mistaken for problems with the AI model itself. When serving software leaks or corrupts data, users often assume the model is hallucinating, masking the true source of the problem.
The findings have broad implications as organizations increasingly deploy LLMs for customer service, data analysis and internal operations. What began as a classroom project in an advanced course taught by Liu evolved into research with implications for AI systems operating at global scale.
“What Yunze's work with GRIEF proves is that if we don't secure the serving infrastructure, it doesn't matter how safe the underlying AI model is,” Liu said. “A secure model on an insecure serving layer is still a compromised system.”
—Story by Melissa Brachfeld, UMIACS communications group