Sensing and Actuating Technologies for Immersive Surgical Training: A Focus on VR/AR and Haptics

Project Description

This thesis focuses on developing immersive technologies for medical training, specifically on sensing/recording and actuating/rendering systems. The project is part of the European project titled  ImmersiveSurgicalEdu , which aims to create an advanced surgical training platform using Augmented Reality (AR) , Virtual Reality (VR) , and haptics to deliver realistic and interactive simulations.

1) Sensing and Recording:

This aspect involves capturing real-time data from users and environments during surgical simulations.

• Objective: Implement sensor technologies (eg, motion capture, glove-based sensors, depth cameras) to record hand movements, forces, tool use, and physiological parameters.
• Key Focus: Accurate tracking of human interactions with instruments and environment, ensuring low latency and high precision.
• Outcome: A fully integrated system that captures user actions and feeds them into a virtual surgical environment in real-time.

2) Actuating and Rendering:

This aspect focuses on providing realistic feedback via haptics, visual and auditory systems.

• Objective: Develop haptic devices and multimodal feedback systems that simulate the tactile, auditory, and visual elements of surgery.
• Key Focus: Creating accurate force feedback and synchronized multi-sensory experiences.
• Outcome: A system that provides realistic haptic and visual/auditory feedback, enhancing the training experience.

Research Methodology:

• Literature Review: Examine existing AR/VR and haptic technologies.
• Prototyping: Develop a prototype integrating sensors and feedback systems.
• Testing: Evaluate the system's performance and user experience in simulated surgeries.

Learning Outcomes:

• Gain expertise in AR/VR, haptics, and sensor technologies.
• Develop skills in prototyping and testing immersive training systems.
• Problem-solving in sensor accuracy, latency reduction, and feedback realism.

Ideal Candidate:

Suitable for students in Mechatronics  or Computer Science with programming skills (C++, Python, Unity/Unreal). Interest in AR/VR and human-computer interaction is a plus.

NOTE: the students will be involved with project partners from Italy and  Lithuania, meeting experts and establishing contacts with companies, ie,  WEART haptic solutions .