Video-based fully automatic assessment of open surgery suturing skills

Adam Goldbraikh, Anne Lise D’Angelo, Carla M. Pugh, Shlomi Laufer

Research output: Contribution to journalArticlepeer-review

Abstract

Purpose: The goal of this study was to develop a new reliable open surgery suturing simulation system for training medical students in situations where resources are limited or in the domestic setup. Namely, we developed an algorithm for tools and hands localization as well as identifying the interactions between them based on simple webcam video data, calculating motion metrics for assessment of surgical skill. Methods: Twenty-five participants performed multiple suturing tasks using our simulator. The YOLO network was modified to a multi-task network for the purpose of tool localization and tool–hand interaction detection. This was accomplished by splitting the YOLO detection heads so that they supported both tasks with minimal addition to computer run-time. Furthermore, based on the outcome of the system, motion metrics were calculated. These metrics included traditional metrics such as time and path length as well as new metrics assessing the technique participants use for holding the tools. Results: The dual-task network performance was similar to that of two networks, while computational load was only slightly bigger than one network. In addition, the motion metrics showed significant differences between experts and novices. Conclusion: While video capture is an essential part of minimal invasive surgery, it is not an integral component of open surgery. Thus, new algorithms, focusing on the unique challenges open surgery videos present, are required. In this study, a dual-task network was developed to solve both a localization task and a hand–tool interaction task. The dual network may be easily expanded to a multi-task network, which may be useful for images with multiple layers and for evaluating the interaction between these different layers.

Original languageEnglish
Pages (from-to)437-448
Number of pages12
JournalInternational journal of computer assisted radiology and surgery
Volume17
Issue number3
DOIs
StatePublished - Mar 2022

Keywords

  • Motion metrics
  • Surgical Simulation
  • Surgical video data
  • Tool localization

All Science Journal Classification (ASJC) codes

  • Surgery
  • Biomedical Engineering
  • Radiology Nuclear Medicine and imaging
  • Computer Vision and Pattern Recognition
  • Computer Science Applications
  • Health Informatics
  • Computer Graphics and Computer-Aided Design

Fingerprint

Dive into the research topics of 'Video-based fully automatic assessment of open surgery suturing skills'. Together they form a unique fingerprint.

Cite this