Automatic assessment of performance in the FLS trainer using computer vision

Aviad Lazar, Gideon Sroka, Shlomi Laufer

Research output: Contribution to journalArticlepeer-review


Background: Fundamentals of Laparoscopic Surgery (FLS) box trainer is a well-accepted method for training and evaluating laparoscopic skills. It mandates an observer that will measure and evaluate the trainee’s performance. Measuring performance in the Peg Transfer task includes time and penalty for dropping pegs. This study aimed to assess whether computer vision (CV) may be used to automatically measure performance in the FLS box trainer. Methods: Four groups of metrics were defined and measured automatically using CV. Validity was assessed by dividing participants to 3 groups of experience levels. Twenty-seven participants were recorded performing the Peg Transfer task 2–4 times, amounting to 72 videos. Frames were sampled from the videos and labeled to create an image dataset. Using these images, we trained a deep neural network (YOLOv4) to detect the different objects in the video. We developed an evaluation system that tracks the transfer of the triangles and produces a feedback report with the metrics being the main criteria. The metric groups were Time, Grasper Movement Speed, Path Efficiency, and Grasper Coordination. The performance was compared based on their last video (3 participants were excluded due to technical issues). Results: The ANOVA tests show that for all metrics except one, the variance in performance can be explained by the experience level of participants. Senior surgeons and residents significantly outperform students and interns on almost every metric. Senior surgeons usually outperform residents, but the gap is not always significant. Conclusion: The statistical analysis shows that the metrics can differentiate between the experts and novices performing the task in several aspects. Thus, they may provide a more detailed performance analysis than is currently used. Moreover, these metrics calculation is automatic and relies solely on the video camera of the FLS trainer. As a result, they allow independent training and assessment.

Original languageEnglish
Pages (from-to)6476-6482
Number of pages7
JournalSurgical Endoscopy
Issue number8
StatePublished - Aug 2023


  • Artificial intelligence
  • Clinical Competence
  • Computer Simulation
  • Computer vision
  • Computers
  • FLS
  • Humans
  • Laparoscopy
  • Laparoscopy/methods
  • Skill assessment
  • Task Performance and Analysis
  • User-Computer Interface

All Science Journal Classification (ASJC) codes

  • Surgery


Dive into the research topics of 'Automatic assessment of performance in the FLS trainer using computer vision'. Together they form a unique fingerprint.

Cite this