Validating artificial intelligence based surgical skill assessment for robotic hiatal hernia repair using a deep learning vision transformer model

The National Institute of Biomedical Imaging and Bioengineering awarded Cedars-Sinai Medical Center $154,038 on October 22, 2026, under the Discovery and Applied Research for Technological Innovations to Improve Human Health program (CFDA 93.286) to validate an artificial intelligence-based surgical skill assessment tool for robotic hiatal hernia repair.

The recipient will adapt and validate the Surgical AI System (SAIS), a deep learning vision transformer model previously developed for robotic urologic procedures, to assess surgeon performance in robotic hiatal hernia repair. Aim 1 establishes a structured dataset of hiatal hernia repair videos scored using the End-to-End Assessment of Suturing Expertise (EASE) framework, a validated tool for evaluating robotic suturing skills. Annotators will evaluate surgical maneuvers, assess inter-rater reliability, and compare EASE scores across surgeon experience levels to validate generalizability. Aim 2 adapts the existing SAIS model to automate EASE scoring in hiatal hernia repair, validates model performance using ten-fold cross-validation, and evaluates agreement between AI-generated and human-generated scores. The work aims to establish foundations for future studies linking surgical skill to patient outcomes and inform development of AI-driven feedback systems for surgical training, credentialing, and quality improvement.

Performance occurs in West Hollywood, California. The project period runs through June 28, 2028.

Abstract

Surgical skill is a critical determinant of patient outcomes, yet current methods for assessing intraoperative performance remain subjective, labor-intensive, and difficult to scale. Advances in artificial intelligence (AI) and computer vision offer an opportunity to develop automated, objective tools for surgical skill assessment. Our group has previously developed the surgical AI system (sais), a deep learning-based vision transformer model capable of evaluating surgical skills in robotic urologic procedures. However, the generalizability of this technology beyond urology has not been explored. This project aims to adapt and validate sais for robotic hiatal hernia repair (rhhr), an ideal test case given its high recurrence rates, technical complexity, and increasing use of robotic platforms. In aim 1, a structured dataset of rhhr videos will be established and scored using the end-to-end assessment of suturing expertise (ease) framework, a validated tool for evaluating robotic suturing skills. Annotators will evaluate surgical maneuvers in rhhr, and inter-rater reliability will be assessed. Ease scores will be compared across surgeon experience levels to validate their generalizability to rhhrs. In aim 2, this ground truth dataset will be used to adapt the existing sais vision transformer model to automate ease scoring in rhhr. Model performance will be validated using ten-fold cross-validation, and agreement between AI- and human-generated ease scores will be evaluated. This work will lay the foundation for future studies investigating the relationship between surgical skill and patient outcomes, and contribute to the long-term goal of developing AI-driven feedback systems to enhance surgical training, credentialing, and quality improvement. This aligns with the national institute of biomedical imaging and bioengineering's mission to drive the development of biomedical technologies that improve healthcare by advancing medical imaging, informatics, and computational modeling. The applicant will conduct this research under the mentorship of the hung lab, which has a strong track record of AI-driven surgical assessment research. The fellowship training plan includes hands-on AI research experience, coursework in deep learning and computer vision, and mentorship from experts in surgery, machine learning, and biostatistics. By the end of the fellowship, the applicant will have developed expertise in AI-driven surgical analysis, bridging engineering and surgery to create scalable, objective methods for evaluating intraoperative performance. By providing protected time for training, research, and career development, this grant will support the applicant's transition to becoming an independent surgeon-scientist at the forefront of AI-driven surgical research.