The University of Maryland will lead a new cross-institutional research effort designed to dramatically accelerate mathematical discovery by combining advanced artificial intelligence (AI) with deep human expertise in computer science and mathematics.
As mathematicians tackle increasingly complex questions—from cybersecurity to algorithm design used to verify software in aerospace, healthcare, and autonomous vehicles—researchers aim to develop AI systems that can reason strategically alongside human experts, helping address problems whose scale and complexity have long limited progress.
To meet this challenge, researchers at Maryland and the Massachusetts Institute of Technology have launched Generative Exploration for Novel Inference in Unsolved Mathematics (GENIUS). The initiative is supported by a $2.6 million award from the Defense Advanced Research Projects Agency (DARPA) through its Exponentiating Mathematics (expMath) program.
The project will develop AI “co-authors” capable of helping mathematicians break down extremely complex problems and uncover new insights.
GENIUS seeks to move AI beyond the current systems toward tools capable of deeper, more structured reasoning, said Mohammad Hajiaghayi, the Jack and Rita G. Minker Professor of Computer Science at Maryland and lead principal investigator.
“We are capturing the hidden scaffolding behind expert human reasoning—including failed attempts and decomposition strategies—and turning that into trainable structure for AI systems,” said Hajiaghayi, who has an appointment in the University of Maryland Institute for Advanced Computer Studies (UMIACS) and is also active in the Artificial Intelligence Interdisciplinary Institute at Maryland (AIM). “If successful, this will fundamentally change how complex proofs and new mathematical insights are generated.”
Joining Hajiaghayi on the project are Reza Ghanadan, research professor in Maryland’s Institute for Systems Research and executive director of the Innovations in AI initiative at UMD’s A. James Clark School of Engineering; Erik Demaine, professor of computer science at MIT; and Ankur Moitra, the Norbert Wiener Professor of Mathematics at MIT. Together, the team brings expertise in algorithms, theoretical computer science, mathematics, machine learning and AI systems.
At the heart of the DARPA-funded effort is a new “neuro-symbolic reasoning engine,” which will embed mathematical abstractions and expert problem-solving strategies directly into large language models.
The system is designed to perform strategic reasoning rather than simple pattern recognition, tackling two longstanding bottlenecks in professional mathematics: decomposing complex problems into reusable steps and translating intuitive reasoning into machine-verifiable formal proofs.
Because mathematicians often take the shortcut of omitting intermediate steps that experts can infer—while computers require every deduction to be fully specified—the GENIUS team will formalize each step of human reasoning into modular components that the AI can reuse, creating a verifiable record of problem-solving.
“Our goal is to build AI systems that don’t just generate answers but learn to reason with structure and strategy,” Ghanadan said. “By embedding symbolic abstractions directly into the model architecture, we aim to create AI collaborators that can operate over long reasoning horizons and contribute meaningfully to deep mathematical discovery.”
Eight to 10 graduate students from Maryland and MIT will work on the project. Among them is Arshia Soltani, a first-year doctoral student in computer science at Maryland.
“For me, the most exciting aspect of the GENIUS project is its end goal,” Soltani said. “The key feature is the possibility that computers could push the boundaries of theoretical computer science forward entirely on their own. I’m also inspired by the exponential growth this could bring to the field.”
The researchers believe the DARPA award underscores the deep level of foundational research, innovation and execution capacity for AI that is being built at Maryland, and they expect broader participation from other faculty and students as their research agenda grows.
“It positions us at the forefront of the next generation of AI for mathematics,” Hajiaghayi said. “By combining rigorous theory with scalable AI systems, we aim to create a foundation for AI–human collaboration that can tackle some of the most challenging problems in science and engineering.”
—Story by Melissa Brachfeld, UMIACS communications group