DOMINO: Domain-Aware Model Calibration in Medical Image Segmentation

Abstract

Model calibration measures the agreement between the predicted probability estimates and the true correctness likelihood. Proper model calibration is vital for high-risk applications. Unfortunately, modern deep neural networks are poorly calibrated, compromising trustworthiness and reliability. Medical image segmentation particularly suffers from this due to the natural uncertainty of tissue boundaries. This is exasperated by their loss functions, which favor overconfidence in the majority classes. We address these challenges with DOMINO, a domainaware model calibration method that leverages the semantic confusability and hierarchical similarity between class labels. Our experiments demonstrate that our DOMINO-calibrated deep neural networks outperform non-calibrated models and state-of-the-art morphometric methods in head image segmentation. Our results show that our method can consistently achieve better calibration, higher accuracy, and faster inference times than these methods, especially on rarer classes. This performance is attributed to our domain-aware regularization to inform semantic model calibration. These findings show the importance of semantic ties between class labels in building confidence in deep learning models. The framework has the potential to improve the trustworthiness and reliability of generic medical image segmentation models. The code for this article is available at: https://github.com/lab-smile/DOMINO.

DOMINO: Domain-aware loss for deep learning calibration

“…So far, DOMINO has been added to segmentation applications in T1-weighted Magnetic Resonance Images (T1-MRIs) and the Cityscapes dataset [ 10 ]. Our results in T1-MRIs are featured in Stolte et al [ 9 ]. Our classification studies have been on the MEDNIST [ 11 ], MNIST [ 12 ], and FashionMNIST [ 13 ] datasets.…”

“…The scheme does not need to be symmetric; for instance, a disease severity model could give step-wise increases in penalization such as giving higher weight to false negatives. Our prior works [ 9 ] use two main W schemes that can be broadly categorized as machine-level confusion (confusion matrix method (DOMINO-CM)) or expert-guided groupings (hierarchical class method (DOMINO-HC)).…”

“…Our prior experiments [ 9 ] show that DOMINO can improve both accuracy and calibration metrics across many classification and segmentation tasks. DOMINO is easy to implement in existing projects via a quick addition to the loss function.…”

“…Both of our methods increases the accuracy even further. Both DOMINO-CM and DOMINO-HC also improve segmentation accuracy [ 9 ].…”

“…Yet, most modern deep learning models are poorly calibrated. Hence, DOMINO [ 9 ] was developed to calibrate deep learning models. The DOMINO loss function that computes penalties for incorrect classifications based on class-wise similarities.…”