On the objectivity, reliability, and validity of deep learning enabled bioimage analyses

Abstract

Bioimage analysis of fluorescent labels is widely used in the life sciences. Recent advances in deep learning (DL) allow automating time-consuming manual image analysis processes based on annotated training data. However, manual annotation of fluorescent features with a low signal-to-noise ratio is somewhat subjective. Training DL models on subjective annotations may be instable or yield biased models. In turn, these models may be unable to reliably detect biological effects. An analysis pipeline integrating data annotation, ground truth estimation, and model training can mitigate this risk. To evaluate this integrated process, we compared different DL-based analysis approaches. With data from two model organisms (mice, zebrafish) and five laboratories, we show that ground truth estimation from multiple human annotators helps to establish objectivity in fluorescent feature annotations. Furthermore, ensembles of multiple models trained on the estimated ground truth establish reliability and validity. Our research provides guidelines for reproducible DL-based bioimage analyses.

Precise measurement of nanoscopic septin ring structures in deep learning-assisted quantitative superresolution microscopy

“…It also resulted in very low error in ring size determination. Our data agree with previous observations that consensus or majority voting models often perform better than models based on individual annotators 48 .…”

HRMAn 2.0: Next‐generation artificial intelligence–driven analysis for broad host–pathogen interactions

“…It allows large numbers of specialists and non-specialists to annotate data for training of CNNs (Figure 3f), by establishing a consensus annotation for each image in the dataset thereby reducing bias in training datasets (>130,000 individual annotations for >8,400 images, as of April 20, 2021) (Pelt, 2020). Machine learning algorithms trained using public-and expert-consensus have proven successful and can accurately complete tasks such as image classification in an unbiased manner (Segebarth et al, 2020;Spiers et al, 2020). 2.6 | New example analysis applications…”

Bioimage Analysis and Cell Motility

“…Indeed, ANNs are now blooming in image segmentation; 96 they are promising because they can be extrapolated almost exhaustively to all different imaging modalities and conditions, but they need to be trained case by case in a (progressively less) laborious process that assumes collected data can be correctly annotated and are representative of future data. 97 , 98 This effort has been especially rewarding to traditionally hard problems, notably to the segmentation of brightfield images during cell-cell contacts. 64…”