Abstract
Most minimally invasive procedures are still performed with hand-held laparoscopic instruments, yet only the endoscopic video is retained; the instrument motion that expresses surgical skill, and that could support skill assessment and robot learning, is lost. Pose from video alone remains millimeters to centimeters off, and an instrument-mounted inertial measurement unit (IMU) cannot rely on its magnetometer, whose field changed with tool pose and between sessions in our measurements. We present a surgical instrument-state logger that clips onto a conventional instrument without modifying the part that enters the patient and fuses a six-axis IMU and a time-of-flight (ToF) rangefinder with a markerless camera in an error-state Kalman filter under the remote center of motion (RCM) of the trocar. Heading comes from the shaft silhouette, segmented by a U-Net, in place of the magnetometer: the rotation-angle error is 0.200°, against 3.58° from the accelerometer and magnetometer alone. Against a Franka Research 3 manipulator, and without alignment to it, the displacement error over 300 translation trials was 1.21mm RMS and the relative-rotation error over 180 rotation trials 0.34° RMS. On continuous trajectories, tracked and displayed in real time, the absolute tip error was 1.22mm (programmed) and 3.04mm (teleoperated) after post-hoc tuning of three filter parameters, and the full fusion beat every sensor subset. Because the estimator uses no magnetic measurement, its accuracy does not rely on an undisturbed field. The same clip-on device could thus record metric tip trajectories during routine hand-held laparoscopy, while displaying the insertion depth and attitude that are hidden once the instrument is inside the patient.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Lee, J., Yee, D., Oh, J., Lee, J., Seo, Y., Jeong, S., Kim, M., Shim, S., Noh, Y., Choi, H., Kong, Y., & Kong, H. J. (2026). Recording Hand-Held Laparoscopic Instrument Motion in the Operating Room: Magnetometer-Free Fusion of Inertial, Range and Visual Sensing. https://omanscience.com/en/articles/recording-hand-held-laparoscopic-instrument-motion-in-the-operating-room-magnetometer-free-fusion-of-inertial-range-and-visual-sensing
MLA 9
Lee, Jiyul, et al. "Recording Hand-Held Laparoscopic Instrument Motion in the Operating Room: Magnetometer-Free Fusion of Inertial, Range and Visual Sensing." https://omanscience.com/en/articles/recording-hand-held-laparoscopic-instrument-motion-in-the-operating-room-magnetometer-free-fusion-of-inertial-range-and-visual-sensing.
Chicago (author–date)
Lee, Jiyul, Dongho Yee, Juahn Oh, Jinseok Lee, Yechan Seo, Seong Jeong, Minsung Kim, Seonho Shim, Younghoon Noh, Hyuk Choi, Youngbin Kong, and Hyoun-Joong Kong. 2026. "Recording Hand-Held Laparoscopic Instrument Motion in the Operating Room: Magnetometer-Free Fusion of Inertial, Range and Visual Sensing." https://omanscience.com/en/articles/recording-hand-held-laparoscopic-instrument-motion-in-the-operating-room-magnetometer-free-fusion-of-inertial-range-and-visual-sensing.
Harvard
Lee, J., Yee, D., Oh, J., Lee, J., Seo, Y., Jeong, S., Kim, M., Shim, S., Noh, Y., Choi, H., Kong, Y. and Kong, H. J. (2026) 'Recording Hand-Held Laparoscopic Instrument Motion in the Operating Room: Magnetometer-Free Fusion of Inertial, Range and Visual Sensing', Available at: https://omanscience.com/en/articles/recording-hand-held-laparoscopic-instrument-motion-in-the-operating-room-magnetometer-free-fusion-of-inertial-range-and-visual-sensing.
Vancouver
Lee J, Yee D, Oh J, Lee J, Seo Y, Jeong S, et al. Recording Hand-Held Laparoscopic Instrument Motion in the Operating Room: Magnetometer-Free Fusion of Inertial, Range and Visual Sensing. https://omanscience.com/en/articles/recording-hand-held-laparoscopic-instrument-motion-in-the-operating-room-magnetometer-free-fusion-of-inertial-range-and-visual-sensing
IEEE
J. Lee, D. Yee, J. Oh, J. Lee, Y. Seo, S. Jeong, M. Kim, S. Shim, Y. Noh, H. Choi, Y. Kong, and H. J. Kong, "Recording Hand-Held Laparoscopic Instrument Motion in the Operating Room: Magnetometer-Free Fusion of Inertial, Range and Visual Sensing," https://omanscience.com/en/articles/recording-hand-held-laparoscopic-instrument-motion-in-the-operating-room-magnetometer-free-fusion-of-inertial-range-and-visual-sensing.