Visual interpretability of image-based classification models by generative latent space disentanglement applied to in vitro fertilization

Abstract

The success of deep learning in identifying complex patterns exceeding human intuition comes at the cost of interpretability. Non-linear entanglement of image features makes deep learning a “black box” lacking human meaningful explanations for the models’ decision. We present DISCOVER, a generative model designed to discover the underlying visual properties driving image-based classification models. DISCOVER learns disentangled latent representations, where each latent feature encodes a unique classification-driving visual property. This design enables “human-in-the-loop” interpretation by generating disentangled exaggerated counterfactual explanations. We apply DISCOVER to interpret classification of in-vitro fertilization embryo morphology quality. We quantitatively and systematically confirm the interpretation of known embryo properties, discover properties without previous explicit measurements, and quantitatively determine and empirically verify the classification decision of specific embryo instances. We show that DISCOVER provides human-interpretable understanding of “black-box” classification models, proposes hypotheses to decipher underlying biomedical mechanisms, and provides transparency for the classification of individual predictions.

Interpretable representation learning for 3D multi-piece intracellular structures using point clouds

“…Deep learning for feature extraction has been shown to be powerful in the context of cell biology, in particular, for analyzing images in 2D [61][62][63] . Despite its success, feature interpretability and generalizability to unseen image data continues to be a major challenge 64,65 . To alleviate some of these problems, it has been shown that imposing additional constraints corresponding to prior biological knowledge to models helps to reduce the space of admissible solutions and improve the likelihood that the learned features can be useful for scientific discovery 66 .…”