Automated machine learning assessment and feedback for the Fundamentals of Laparoscopic Surgery peg transfer task
Year: 2026
Type: article
Abstract: Aim: Whereas computer vision has advanced the capabilities of Fundamentals of Laparoscopic Surgery (FLS) trainers, real-time coaching using machine learning (ML) remains limited. This retrospective proof-of-concept study evaluated whether an ML model could provide automated FLS pass/fail assessment comparable to faculty, using pre-recorded videos. Methods: Residents performed video-recorded FLS peg transfers scored pass/fail. A training set of 60 videos (51 standard cases, 9 edge-case videos) was used to develop a multi-stage pipeline combining a “you only look once” (YOLOv8n) object detector, and a deterministic rule engine encoding FLS pass/fail criteria. The model was evaluated on 24 resident-disjoint, held-out videos. A faculty majority-vote reference standard was set by five evaluators per video. Model predictions were compared against this reference standard. Results: The ML model classified outcomes accurately in 92% of cases [95%CI: 74%-98%; F1 score = 0.93 (pass positive); Cohen’s κ = 0.83]. Total analysis time was 6 min vs. approximately 56 min of video playback. The model concurrently identified errors in 70% (31/44) of error events, with 97% (30/31) concordance in error classification. Among the 13 remaining events, the model preceded the faculty-annotated timestamp in 8/13 (62%). Real-time instructional prompts were implemented in a heads-up display but not evaluated here. Conclusion: The system provided accurate FLS peg-transfer assessment comparable to faculty while reducing evaluation time. As a retrospective proof-of-concept, it supports automated assessment and, pending prospective validation, could deliver scalable real-time feedback in skills training. External validation in larger, multi-institutional cohorts should precede adoption.
Source: Artificial Intelligence Surgery
DOI: 10.20517/ais.2026.72
Language: English
Subfield: Surgery
Field: Medicine
Domain: Health Sciences
Open Access status: hybrid
OpenAlex ID: W7236867384
