BB-UNet: U-Net With Bounding Box Prior

Abstract

Medical image segmentation is the process of anatomically isolating organs for analysis and treatment. Leading works within this domain emerged with the well-known U-Net. Despite its success, recent works have shown the limitations of U-Net to conduct segmentation given image particularities such as noise, corruption or lack of contrast. Prior knowledge integration allows to overcome segmentation ambiguities. This paper introduces BB-UNet (Bounding Box U-Net), a deep learning model that integrates location as well as shape prior onto model training. The proposed model is inspired by U-Net and incorporates priors through a novel convolutional layer introduced at the level of skip connections. The proposed architecture helps in presenting attention kernels onto the neural training in order to guide the model on where to look for the organs. Moreover, it finetunes the encoder layers based on positional constraints. The proposed model is exploited within two main paradigms: as a solo model given a fully supervised framework and as an ancillary model, in a weakly supervised setting. In the current experiments, manual bounding boxes are fed at inference and as such BB-Unet is exploited in a semi-automatic setting; however, BB-Unet has the potential of being part of a fully automated process, if it relies on a preliminary step of object detection. To validate the performance of the proposed model, experiments are conducted on two public datasets: the SegTHOR dataset which focuses on the segmentation of thoracic organs at risk in computed tomography (CT) images, and the Cardiac dataset which is a mono-modal MRI dataset released as part of the Decathlon challenge and dedicated to segmentation of the left atrium. Results show that the proposed method outperforms state-of-the-art methods in fully supervised learning frameworks and registers relevant results given the weakly supervised domain.

Object Detection Improves Tumour Segmentation in MR Images of Rare Brain Tumours

“…We chose You Only Look Once (YOLO) [ 34 , 35 ] as our object-detection framework. For the segmentation, we benchmarked UNet [ 36 ] and Bounding-Box UNet (BB-UNet) [ 28 ]. It must be noted that both UNet and BB-UNet receive the whole 2D images as input for the training and inference; additionally, BB-UNet receives also an a priori bounding-box used internally to (non-exclusively) focus the learning segmentation process.…”

“…Lempitsky et al [ 27 ] incorporated user-provided bounding-boxes as a way to add topology and shape information to their loss function and applied it to the natural images object segmentation problem. Rosana et al [ 28 ] proposed to feed their UNet architecture with their user-provided bounding-box masks in parallel to their input images. These propositions do not discuss the origin of the bounding-boxes, and while user-provided bounding-boxes can be reliable, automatically detected ones can introduce multiple issues, which we propose to study.…”

Deep Neural Architectures for Medical Image Semantic Segmentation: Review

“…There is a wide range of evaluation metrics used to analyze model behavior in semantic segmentation [115]- [119]. This article discusses those metrics that are reviewed.…”

Semantic Segmentation of Urinary Bladder Cancer Masses from CT Images: A Transfer Learning Approach

“…The contractive part of U-net is a standard down-sampling procedure similar to every CNN architecture with its convolutional and pooling layers [ 43 ]. On the other hand, during the expansive part, an up-sampling procedure is performed [ 44 ].…”