Predicting the Timing of Camera Movements From the Kinematics of Instruments in Robotic-Assisted Surgery Using Artificial Neural Networks

Hanna Kossowsky, Ilana Nisky

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

Robotic surgeries offer many benefits, however do not allow for simultaneous control of the endoscopic camera and the surgical instruments. This leads to frequent interruptions as surgeons adjust their viewpoints. Autonomous camera control could help overcome this challenge. We propose a predictive approach for anticipating when camera movements will occur using artificial neural networks. We used kinematic data of surgical instruments from robotic surgical training. We split the data into segments, and labeled if each segment immediately preceded a camera movement or did not. Due to the large class imbalance, we trained an ensemble of networks on balanced sub-sets of the data. We found that the instruments’ kinematics can be used to predict when camera movements will occur, and evaluated the performance on different segment durations and ensemble sizes. We also studied how much in advance upcoming camera movements can be predicted, and found that predicting camera movements up to 0.5 s in advance led to only a small decrease in performance relative to predicting imminent camera movements. These results serve as a proof-of-concept for predicting the timing of camera movements in robotic surgeries and suggest that an autonomous camera controller for robotic surgeries may someday be feasible.

Original languageEnglish
Pages (from-to)391-402
Number of pages12
JournalIEEE Transactions on Medical Robotics and Bionics
Issue number2
DOIs
StateAccepted/In press - 1 Jan 2022

Keywords

  • artificial neural networks
  • Cameras
  • endoscopic camera
  • Instruments
  • kinematics
  • Kinematics
  • RAMIS
  • Robot vision systems
  • Robots
  • Surgery
  • surgical instruments.
  • Timing

ASJC Scopus subject areas

  • Computer Science Applications
  • Artificial Intelligence
  • Human-Computer Interaction
  • Control and Optimization
  • Biomedical Engineering

Fingerprint

Dive into the research topics of 'Predicting the Timing of Camera Movements From the Kinematics of Instruments in Robotic-Assisted Surgery Using Artificial Neural Networks'. Together they form a unique fingerprint.

Cite this