Imagine asking an AI chatbot for health advice and getting conflicting guidance—or turning to a chatbot in a crisis only to receive unclear instructions. Confusing or inconsistent AI isn’t just frustrating; it can put people’s health and safety at risk.
Researchers in the Computational Linguistics and Information Processing (CLIP) Lab—including Jordan Boyd-Graber, a professor of computer science with an appointment in the University of Maryland Institute for Advanced Computer Studies (UMIACS); and Vanessa Frias-Martinez, a professor in the College of Information with an appointment in UMIACS—are tackling these risks by building AI systems that people can trust.
One study, "Discrepancy Detection at the Data Level: Toward Consistent Multilingual Question Answering," examines how answers can differ across languages even when factually correct. Multilingual QA systems face the dual challenge of ensuring factual accuracy while respecting cultural relevance: a correct answer in one language may not meet expectations in another due to differences in local customs, data availability, or nuance.
To address this, the team—including researchers from UMD and the Universidad Carlos III de Madrid in Spain—created a system that proactively identifies discrepancies across languages before they appear in AI-generated answers. Dubbed MIND (Multilingual Inconsistent Notion Detection), the system aligns documents from different languages in a shared conceptual space, compares interpretations, and flags factual or culturally divergent information. For example, guidance on childbirth practices can vary by region, and MIND highlights these differences so users can trust the information.
While not fully hands-off, MIND focuses human attention mainly on flagged discrepancies, reducing the effort needed to review answers. Tested on bilingual maternal and infant health data—and other domains—MIND reliably identifies inconsistencies. By highlighting cultural differences tied to language, the system could also help reduce bias and better support underrepresented communities. To encourage broader research in culturally aware AI, the team also released a dataset of annotated bilingual questions for other researchers to build on.
The work was led by Lorena Calvo-Bartolomé, a Ph.D. student at Universidad Carlos III de Madrid, who was a visiting researcher at UMD in Fall 2024 under the supervision of Boyd-Graber.
The collaboration was part of the “Rosie” project to build bilingual chatbots that support new and expectant mothers in both English and Spanish. Other UMD researchers on the Rosie team include Valérie Aldana, a third-year doctoral student in the Department of Behavioral and Community Health; and Karla Cantarero, who earned her bachelor of science in public health in May. Co-authors from the Universidad Carlos III de Madrid include Alonso Madroñal de Mesa, who recently graduated with degrees in data science, engineering and telecommunications engineering; and Jerónimo Arenas-García, a professor of signal theory and communications.
While MIND addresses cultural consistency in everyday information, a second study, "A Dynamic Fusion Model for Consistent Crisis Response," focuses on high-stakes crisis communication, where inconsistent responses can reduce public trust and even endanger lives. People in crisis—like navigating natural disasters or active shooters—often turn to social networks for information, support and assistance, especially when official sources are overwhelmed or slow to respond. While some information from the public is helpful, inaccurate or conflicting posts—like the confusion over shelter requirements on Twitter during Hurricane Irma —can create real risks.
Direct communication from government agencies and disaster-relief non-governmental organizations is critical, but these organizations often can’t respond promptly to every individual. A one-size-fits-all approach is rarely effective.
To address this, the team—researchers from UMD, the University of North Texas, and the University of Arizona—developed a fusion-based framework that uses AI to generate responses that are consistently professional, actionable, and relevant. The system evaluates multiple candidate messages and blends the strongest elements to produce clear, reliable guidance.
Testing shows the framework delivers higher-quality responses across a range of emergencies, outperforming existing methods and helping users understand what to trust and what actions to take during critical moments.
“In high-stakes emergencies, inconsistency in crisis communication can erode trust and obstruct effective response,” says Frias-Martinez, a co-author on the paper. “Our fusion-based framework demonstrates that AI can synthesize the most reliable elements of multiple candidate messages to produce responses that are uniform in professionalism, clarity, and relevance.”
She adds that the framework can support government and nonprofit organizations in delivering high-quality guidance across crisis scenarios.
Other co-authors on the paper include Xiaoying Song, a Ph.D. student in information at the University of North Texas; Anirban Saha Anik, a graduate student in data science at the University of North Texas; Eduardo Blanco, an associate professor of computer science at the University of Arizona; and Lingzi Hong, an associate professor of data science at the University of North Texas who earned her Ph.D. in information studies from UMD in 2019.
Together, these studies show how UMD researchers are building AI that provides consistent, understandable guidance across languages, cultures, and urgent circumstances—ultimately keeping users safer, better informed, and more confident in the decisions they make.
The papers were two of 19 that were presented by CLIP researchers at the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP), held in Suzhou, China from November 4–9. They highlight UMD’s work in creating AI that not only gives correct answers, but communicates in ways people can trust and act on.
—Story by Melissa Brachfeld, UMIACS communications group