A semi-supervised Teacher-Student framework for surgical tool detection and localization

Abstract

ABSTRACT Surgical tool detection in minimally invasive surgery is an essential part of computer-assisted interventions. Current approaches are mostly based on supervised methods requiring large annotated datasets. However, labelled datasets are often scarce. Semi-supervised learning (SSL) has recently emerged as a viable alternative showing promise in producing models retaining competitive performance to supervised methods. Therefore, this paper introduces an SSL framework in the surgical tool detection paradigm, which aims to mitigate training data scarcity and data imbalance problems through a knowledge distillation approach. In the proposed work, we train a model with labelled data which initialises the Teacher-Student joint learning, where the Student is trained on Teacher-generated pseudo-labels from unlabelled data. We also propose a multi-class distance with a margin-based classification loss function in the region-of-interest head of the detector to segregate the foreground-background region effectively. Our results on m2cai16-tool-locations dataset indicates the superiority of our approach on different supervised data settings (1%, 2%, 5% and 10% of annotated data) where our model achieves overall improvements of 8%, 12%, and 27% in mean average precision on 1% labelled data over the state-of-the-art SSL methods and the supervised baseline, respectively. The code is available at https://github.com/Mansoor-at/Semi-supervised-surgical-tool-detection.