الملخص
Current robot-assisted minimally-invasive surgery (RMIS) platforms provide a fixed console for the surgeon to view stereo endoscopic images and teleoperate instruments inside the patient. Several researchers have proposed the use of a head-mounted display (HMD) as a portable console, with video pass-through rendering of the endoscope images which, like the fixed console, restricts the operator to a single endoscopic viewpoint and limits depth perception. We present a digital twin-driven virtual reality (VR) teleoperation platform, where the digital twin is created from markerless perception of the surgical environment and streamed for display on the HMD. This overcomes the limitations of video pass-through by providing multi-view rendering and natural motion-parallax cues, enabling decoupling of the user's hand posture from strict instrument alignment. The system utilizes VR hand controllers to increase the teleoperation workspace and to improve the robustness and stability of instrument control compared to the hand tracking approach adopted by most prior systems. A 15-participant user study on the da Vinci Research Kit (dVRK) shows that our VR platform significantly outperforms a state-of-the-art HoloLens 2 mixed reality baseline, reducing path length by 86% and jerk by 95%, while achieving depth perception confidence comparable to or exceeding the traditional console across all conditions.
الكلمات المفتاحية
الموضوع
بيانات النشر
- المجلة
- غير متاح
- وصول مفتوح
- وصول مفتوح أخضر
اقتبس هذه المقالة
APA 7
Liu, C., Yu, C., Zhao, H., Ding, H., Wei, H., Munawar, A., Unberath, M., & Kazanzides, P. (2026). Digital Twin-Driven VR Teleoperation with Multi-View Spatial Perception for Surgical Robots. https://omanscience.com/ar/articles/digital-twin-driven-vr-teleoperation-with-multi-view-spatial-perception-for-surgical-robots
MLA 9
Liu, Chang, et al. "Digital Twin-Driven VR Teleoperation with Multi-View Spatial Perception for Surgical Robots." https://omanscience.com/ar/articles/digital-twin-driven-vr-teleoperation-with-multi-view-spatial-perception-for-surgical-robots.
شيكاغو (المؤلف–التاريخ)
Liu, Chang, Chenhao Yu, Honghao Zhao, Hao Ding, Haochen Wei, Adnan Munawar, Mathias Unberath, and Peter Kazanzides. 2026. "Digital Twin-Driven VR Teleoperation with Multi-View Spatial Perception for Surgical Robots." https://omanscience.com/ar/articles/digital-twin-driven-vr-teleoperation-with-multi-view-spatial-perception-for-surgical-robots.
هارفارد
Liu, C., Yu, C., Zhao, H., Ding, H., Wei, H., Munawar, A., Unberath, M. and Kazanzides, P. (2026) 'Digital Twin-Driven VR Teleoperation with Multi-View Spatial Perception for Surgical Robots', Available at: https://omanscience.com/ar/articles/digital-twin-driven-vr-teleoperation-with-multi-view-spatial-perception-for-surgical-robots.
فانكوفر
Liu C, Yu C, Zhao H, Ding H, Wei H, Munawar A, et al. Digital Twin-Driven VR Teleoperation with Multi-View Spatial Perception for Surgical Robots. https://omanscience.com/ar/articles/digital-twin-driven-vr-teleoperation-with-multi-view-spatial-perception-for-surgical-robots
IEEE
C. Liu, C. Yu, H. Zhao, H. Ding, H. Wei, A. Munawar, M. Unberath, and P. Kazanzides, "Digital Twin-Driven VR Teleoperation with Multi-View Spatial Perception for Surgical Robots," https://omanscience.com/ar/articles/digital-twin-driven-vr-teleoperation-with-multi-view-spatial-perception-for-surgical-robots.