ENRICHing Medical Imaging Training Sets Enables More Efficient Machine Learning

Abstract

Deep learning (DL) has been applied with success in proofs of concept across biomedical imaging, including across modalities and medical specialties1-17. Labeled data is critical to training and testing DL models, and such models traditionally require large amounts of training data, straining the limited (human) resources available for expert labeling/annotation. It would be ideal to prioritize labeling those images that are most likely to improve model performance and skip images that are redundant. However, straightforward, robust, and quantitative metrics for measuring and eliminating redundancy in datasets have not yet been described. Here, we introduce a new method, ENRIch (Eliminate Needless Redundancy in Imaging datasets), for assessing image dataset redundancy and test it on a well benchmarked medical imaging dataset3. First, we compute pairwise similarity metrics for images in a given dataset, resulting in a matrix of pairwise similarity values. We then rank images based on this matrix and use these rankings to curate the dataset, to minimize dataset redundancy. Using this method, we achieve similar AUC scores in a binary classification task with just a fraction of our original dataset (AUC of 0.99 +/- 1.35e-05 on 44 percent of available images vs. AUC of 0.99 +/- 9.32e-06 on all available images, p-value 0.0002) and better scores than the same sized training subsets chosen at random. We also demonstrate similar Jaccard scores in a multi-class segmentation task while eliminating redundant images (average Jaccard index of 0.58 on 80 percent of available images vs. 0.60 on all available images). Thus, algorithms that reduce dataset redundancy based on image similarity can significantly reduce the number of training images required, while preserving performance, in medical imaging datasets.