Training data distribution significantly impacts the estimation of tissue microstructure with machine learning

Abstract

Purpose Supervised machine learning (ML) provides a compelling alternative to traditional model fitting for parameter mapping in quantitative MRI. The aim of this work is to demonstrate and quantify the effect of different training data distributions on the accuracy and precision of parameter estimates when supervised ML is used for fitting. Methods We fit a two‐ and three‐compartment biophysical model to diffusion measurements from in‐vivo human brain, as well as simulated diffusion data, using both traditional model fitting and supervised ML. For supervised ML, we train several artificial neural networks, as well as random forest regressors, on different distributions of ground truth parameters. We compare the accuracy and precision of parameter estimates obtained from the different estimation approaches using synthetic test data. Results When the distribution of parameter combinations in the training set matches those observed in healthy human data sets, we observe high precision, but inaccurate estimates for atypical parameter combinations. In contrast, when training data is sampled uniformly from the entire plausible parameter space, estimates tend to be more accurate for atypical parameter combinations but may have lower precision for typical parameter combinations. Conclusion This work highlights that estimation of model parameters using supervised ML depends strongly on the training‐set distribution. We show that high precision obtained using ML may mask strong bias, and visual assessment of the parameter maps is not sufficient for evaluating the quality of the estimates.

Multi‐band‐ and in‐plane‐accelerated diffusion MRI enabled by model‐based deep learning in q‐space and its extension to learning in the spherical harmonic domain

“…By simply matching the q‐space sampling parameters in the biophysical models to the desired q‐space, such training data can be easily generated. Moreover, biophysical models provide a convenient method to generate training data that span a wide range of parameters such as diffusivities and volume fractions and fiber orientations, whereas, the parameter span of data‐driven training may be clustered around certain ranges 24 . This convenient framework also allows for re‐training the neural network to learn a new q‐space manifold by simply modifying the q‐space sampling parameters in the biophysical models.…”

“…Moreover, biophysical models provide a convenient method to generate training data that span a wide range of parameters such as diffusivities and volume fractions and fiber orientations, whereas, the parameter span of datadriven training may be clustered around certain ranges. 24 This convenient framework also allows for re-training the neural network to learn a new q-space manifold by simply modifying the q-space sampling parameters in the biophysical models. One disadvantage of previous data-driven DL methods is the generalizability of the learned information.…”

Reproducibility of the Standard Model of diffusion in white matter on clinical MRI systems

“…Neural networks (Golkov et al, 2016), polynomial regression (Reisert et al, 2017), or random forest (Palombo et al, 2020) have provided useful results in different dMRI applications. The ML approach has been applied to estimate SM parameters by (Reisert et al, 2017), and followed by more recent works (Coelho et al, 2021a;Gyori et al, 2021;de Almeida Martins et al, 2021). Irrespective of the implementation, all these works concluded that ML estimation alone is unable to resolve SM parameter degeneracies, and that a sufficiently rich acquisition protocol is needed.…”

Active-learning-based non-intrusive Model Order Reduction

“…Therefore, the main focus will be improving the distribution of the training data. There are some investigations [13,14] talking about how the distribution of the data could influence the machine learning. However, this issue is hardly discussed in the context of MOR.…”