Using distilled knowledge from large transformer models for efficient identification of cataract surgery steps
Year: 2026
Type: article
Abstract: Introduction Automated analysis of cataract surgery videos can provide objective performance metrics for trainees and support consistent progress tracking. Large vision transformers perform well for medical image tasks, but their size can limit practical deployment. Methods We evaluated whether knowledge distillation can produce an efficient model for cataract surgical step recognition while preserving accuracy. We analyzed 268 resident cataract surgery videos collected across six sites and labeled with 14 surgical steps. A large teacher model pretrained with self-supervision was compared with a smaller student model trained using knowledge distillation, the same student trained without distillation, and a convolutional neural network baseline. Performance was assessed using overall accuracy and area under the receiver operating characteristic curve, and efficiency was assessed using parameter count and floating-point operations per frame. Results The distilled student achieved 87.7% accuracy and an area under the curve of 0.985, comparable to the teacher (87.7% accuracy; 0.970). The distilled student was substantially smaller (21.7 million vs. 85.8 million parameters) and more efficient (4.63 vs. 17.58 GFLOPs per frame). It also outperformed the student trained without distillation (85.5% accuracy; 0.963) and the best convolutional neural network baseline (85.6% accuracy; 0.973). Discussion These findings suggest that knowledge distillation can produce a lightweight, high-performing model for cataract surgical step recognition, reducing computational barriers relevant to deployment of automated feedback tools for surgical training.
Source: Frontiers in Digital Health
Authors: John Xiang(Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing – original draft, Writing – review & editing), Sophia Y. Wang(Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Writing – original draft, Writing – review & editing)
Institution: Stanford University
Language: English
Subfield: Surgery
Field: Medicine
Domain: Health Sciences
Open Access status: gold
Estimated APC paid: $3,077
OpenAlex ID: W7236569709
