When the SPIE Defense + Security conference brought together leading researchers and innovators from across the globe last week, one presenter, a sophomore at John Carroll University, shared his augmented reality system, “HoloPose,” that has the potential to make an impact in rehabilitation.
Wilson Vong ’28, a Computer Science major with minors in Data Science and Business Analytics, and his professor and mentor, Dr. Almabrok Essa, Assistant Professor of Computer Science and Director of the Artificial Intelligence Minor and lab space at JCU, presented their research at the conference on Tuesday, April 28, in National Harbor, Maryland.
Vong’s research was funded through the NSF S-STEM program (NSF DUE #2322446), and the pair’s
conference participation was supported through a collaborative network of campus funding, including the Colleran-Weaver Endowment, the Center for Student Belonging, the Honors Program, and the Office of Academic Affairs.
Vong began this work as part of John Carroll’s Summer Undergraduate Research Fund (SURF) in 2025, when he was still a first-year student, an achievement that underscores the strength of undergraduate research opportunities at John Carroll.
What started as a summer project quickly evolved into an innovative system with real-world healthcare applications.
Turning Research into Real-World Impact
Working alongside Dr. Essa, Vong developed “HoloPose,” an augmented reality (AR) system powered by artificial intelligence.
“By fusing advanced AI with AR, Wil creates cohesive, high-impact solutions that redefine real-time 3D human pose estimation in healthcare,” said Dr. Essa. “His innovative framework for rehabilitation showcases exceptional technical skills, independence, and a visionary approach to technology.”
Using a Microsoft HoloLens headset, the system captures a 2D image of a subject and processes it through an AI model that identifies key body points, such as the nose, shoulders, and hips. From there, geometric analysis translates those points into meaningful health metrics, including shoulder tilt, hip alignment, and neck positioning.
The result is a system that can classify posture into three categories:
- Good – safe
and properly aligned - Warning – likely to cause discomfort or strain
- Risk – requiring corrective action
Designed with rehabilitation in mind, the technology provides real-time, on-device feedback, without the need for external hardware. By projecting visual cues like dots and lines onto the subject’s body and delivering audio and text feedback, the system allows for hands-free interaction and safety in both clinical and home settings.
“AI acts as the 'brain' that figures things out. In this case, it's used to understand the pose or position of a subject,” explained Vong. “The model relies on deep learning to recognize patterns in human poses. Deep learning models consist of many interconnected ‘neurons’ that process input data, such as images, and transform it into simpler, more user-friendly outputs. Without AI, we wouldn't be able to do any of the mathematical calculations. The nice thing about all of this is that the system runs completely on the headset, no external connections or processing! No need for external sensors or off-device processing/inference.”
A Foundation Built Through Mentorship
For Vong, the project is as much about mentorship as it is about innovation.
“I’ve really enjoyed the research Dr. Essa does and the mentorship he provides me,” Vong said. “I started conducting research with him in the second semester of my freshman year. As a result, I’ve been able to apply what I learned in the classroom to a real-world application that can hopefully improve physical therapy by cutting down the amount of time it takes medical professionals to make a diagnosis. Doing this project has definitely given me a springboard into the work I want to do in the future.”
Dr. Essa played a key role in guiding the project from concept to conference-ready research, helping Vong navigate technical challenges and reach critical milestones.
Successfully completing an undergraduate research project was only Vong’s first foray into shaping his career in computer science. After presenting at the SPIE Defense + Security conference and networking with other students and professionals in the field, he will continue his research with Dr. Essa this summer.
