Artificial intelligence-powered surgical copilot in minimally invasive adrenal surgery: Integrated recognition of anatomy, tools, and phases, to updates in surgery

Abstract

Abstract Background This study aimed to develop an artificial intelligence–powered surgical assistant model capable of recognising key anatomical structures, tools, and surgical phases during minimally invasive adrenal surgery, and to assess its clinical feasibility. The system has potential to enhance procedural safety and support training for adrenal operations, which are performed relatively infrequently. Method Video recordings from 20 laparoscopic transabdominal adrenalectomies (10 from each side; 2011–2025) were retrospectively analysed. Surgical frames were annotated for tools, actions, anatomy, and surgical phases to build the dataset. A ResNet-50–based multi-head cross-attention network with dedicated heads for tools, actions, targets, and phases was developed. Laterality was incorporated through feature-wise linear modulation, and class imbalance was handled using weighted loss. Performance was assessed on a balanced test set using top-accuracy and mean average precision with 5-fold cross-validation. Results A total of 10,747 frames were labelled with 53,152 annotations across 9 tool categories, 7 surgical actions, 13 anatomical targets, and 6 phases. The model achieved top-1 accuracies of 67.6% for tools, 86.2% for actions, 70.0% for targets, and 60.3% for phases. Top-3 accuracies increased to 93.8%, 94.2%, 90.9%, and 95.3%, respectively. Mean average precision values were 64.2% for tools, 62.4% for actions, 46.8% for targets, and 66.4% for phases. Conclusion The developed model can successfully recognize tools, actions, and key anatomical structures in adrenal surgery. It has the potential to provide comprehensive analysis of surgical scenes in minimally invasive adrenalectomy, which may help improve safety, reduce complications, and support surgeon training through objective, automated feedback.