A home cook finishes chopping vegetables, pauses for a moment, and reaches for the soy sauce when the recipe called for salt. Small slips like this happen in the brief gap between one step and the next, and for beginners they add up, slowing learning and chipping away at confidence.
AI assistants can already guide people through a task on a screen or by voice. Guiding the body itself, at the moment a mistake is about to happen, calls for a different approach.
To catch these physical mistakes before they happen, University of Maryland researchers created a wearable artificial intelligence system that physically redirects a user mid-motion.
The research is being presented this week at the Association for Computing Machinery Symposium on User Interface Software and Technology (UIST), the premier conference for innovations in human-computer interfaces, in Detroit from November 2-5.
By pairing real-time gazetracking with wearable physical actuators, the UMD system offers humans a predictive safety net — a development that could transform workforce training, rehabilitation, and physical skill learning.
The research was led by Kyungyeon Lee, a sixth-year Ph.D. student in computer science, with Kirti Kishore, a faculty research engineer in the University of Maryland Institute for Advanced Computer Studies (UMIACS); Adrien Ramsamy, who received his master’s degree in computer science in May 2026; Atsushi Kikumoto, assistant professor of psychology; and Jun Nishida, assistant professor of computer science with appointments in UMIACS and the Immersive Media Design program.
The team was driven by a shared goal: expanding AI beyond digital screens and into physical human movement.
“AI has already transformed how we work with information on screens, through text, images and code,” Nishida said. “Many of the skills that matter in daily life and work, however, are physical, such as cooking food, assembling products, or operating equipment.”
Alongside advances in physical AI robots that automate tasks, the team pursued a complementary direction: human augmentation with AI.
“Our work explores a wearable AI that works directly with the human body and perception, understanding what people are doing and helping them improve their own physical skills,” Nishida said. “We call this embodied human-AI collaboration.”
To build that vision, the team designed the system as a head- and chest-worn exosuit. By combining custom eye-tracking glasses, a wrist motion tracker, and a dual-loop vision-language model running on a portable NVIDIA Jetson Thor computer, it can track where a user is looking and predict reaching in real time to intercept mistakes and steer the user toward the correct action.
The system — called ProtoCall — deploys two distinct physical nudging mechanisms. In preemptive attentional nudging, if a user stares at the wrong item for more than 800 milliseconds, a lightweight motorized cable attached to a headband gently rotates the user's head toward the correct object. In preemptive reaching nudging, if a user begins reaching toward the wrong item, a chest-mounted motor rapidly pulls the arm back within 113 milliseconds, stopping the hand mid-air before it touches the incorrect item.
In a study with 16 participants performing a 15-step cooking-themed sequential task, all three forms of physical feedback led to fewer errors than practicing without feedback, both during initial practice and when participants later performed the task on their own. Among them, errors decreased from post-error correction to wrist nudges to head nudges, and head nudges produced significantly fewer errors than both other approaches. Participants who received preemptive nudges also reported higher confidence and greater awareness of their errors than those corrected after the fact.
The team also uncovered a key trade-off between head nudges and wrist nudges. Head nudges provided richer spatial context, helping users build a stronger mental map of the entire sequence and achieving the highest learning accuracy. Wrist nudges, on the other hand, offered faster physical intervention and preserved the user’s sense of control, though they primarily communicated what not to do.
Based on these insights, the researchers recommend tailoring physical feedback to skill levels, using head nudges to guide complete beginners through new spatial workflows, and switching to subtle wrist nudges for intermediate users who need fast error prevention without compromising their pace.
Nishida said the most immediate practical application for the technology is workforce training in industrial manufacturing, where new workers must quickly learn long, complex assembly sequences with minimal mistakes. Other potential applications include step-by-step cooking, operating machinery, and helping patients relearn motor functions during physical rehabilitation.
ProtoCall builds on the team's earlier work, FIXical I/O, presented at the Association for Computing Machinery Conference on Human Factors in Computing Systems (CHI) earlier this year and also led by Lee. That system used a hand exoskeleton to sense finger errors in real time and physically correct them while people learned key sequences. ProtoCall extends the same idea from the fingers to the arm and head, and from tapping keys to multi-step tasks in the physical world.
At UIST this week, the team is also presenting a live demonstration that brings both systems together, allowing attendees to experience preemptive physical feedback across the fingers, arms and head firsthand.
For the researchers, systems like ProtoCall represent a broader fundamental shift in how AI supports human capability.
“The same technologies making machines more capable can also help people grow,” Nishida said. “We envision a future in which humans and machines evolve together.”
—Story by Melissa Brachfeld, UMIACS communications group
See more work on haptics, exoskeletons and even muscle stimulation at the Embodied Dynamics Laboratory.