Virtual Mouse Brain Histology from Multi-contrast MRI via Deep Learning

Abstract

words):1 H MRI maps brain anatomy and pathology non-invasively through contrasts generated by exploiting inhomogeneities in tissue micro-environments. Inferring histopathological information from MRI findings, however, remains challenging due to the absence of direct links between MRI signals and specific tissue compartments. Here, we show that convolutional neural networks, developed using coregistered multi-contrast MRI and histological data of the mouse brain, can generate virtual histology from MRI results. Our networks provide maps that mirror histological stains for axons and myelin with enhanced specificity compared to existing MRI markers. Furthermore, by introducing random perturbations to the inputs, the relative contribution of each MRI contrast within the networks can be estimated and guide the optimization of MRI acquisition. We anticipate our method to be a starting point for translation of MRI results into easy-to-understand virtual histology for neurobiologists and provide resources for developing novel MRI contrasts. Introduction:Magnetic resonance imaging (MRI) is one of a few techniques that can image the brain non-invasively and without ionizing radiation. This advantage is further augmented by a large and still increasing array of versatile tissue contrasts (e.g., diffusion 1 , magnetization transfer 2 , and manganese enhanced MRI 3 ).While the rich tissue contrasts provide unparalleled insights into brain structure and function at the macroscopic level 4 , inferring the spatial organization of microscopic structures (e.g. axons and myelin) and their integrity in the brain based on MRI findings remains an ill-posed inverse problem. Aside from the resolution differences, most MRI contrasts are not tied to specific cellular structures and therefore susceptible to confounding factors, especially under pathological conditions. As a result, even though MRI has been widely used to detect certain neuropathology (e.g. ischemic stroke and demyelination), uncertainty arises when the exact pathological events and their severities need to be determined 5-7 . The lack of specificity hinders the direct translation of MRI findings into histopathology and limits its diagnostic value.Tremendous efforts have been devoted to discover new mechanisms to amplify the affinity of MRI signals to unique physical and chemical properties of target cellular structures and, by doing so, generate new contrasts with improved sensitivity and specificity. Recent progress in multi-modal MRI promises enhanced specificity by integrating multiple MRI contrasts that target distinct aspects of a particular

Using Mesoscopic Tract-Tracing Data to Guide the Estimation of Fiber Orientation Distributions in the Mouse Brain from Diffusion MRI

“…Here, we kept the 3x3x3 patch size unchanged because we assumed that the relationship between dMRI signals and axonal bundles in the same voxel should be local and independent of neighboring voxels. Different from (Lin et al, 2019), there was residual mismatches between input dMRI data and target TOD maps, which can be accommodated by the 3x3x3 patch size as suggested by our previous report (Liang et al, 2022). We increased the training data to 3 million patches to avoid over-fitting.…”

“…Here, we trained out network using small 3x3x3 patches instead of entire images because we assumed the relationship between dMRI signals and underlying axonal pathways to be strictly local (i.e. MRI signals are the ensemble average of spins within each voxel and do not depend on neighboring voxels) and would like to limit the amount of spatial information available to the network, in a similar fashion as our previous study connecting MRI signals and histology (Liang et al, 2022). As a result, a limited number of typical dMRI data, instead of thousands as required in typical deep learning studies, can provides sufficient instances to train the deep learning network to resolve the orientation of certain axonal pathways in gray matter.…”

“…Here, we trained out network using small 3x3x3 patches instead of entire images because we assumed the relationship between dMRI signals and underlying axonal pathways to be strictly local (i.e. MRI signals are the ensemble average of spins within each voxel and do not depend on neighboring voxels) and would like to limit the amount of spatial information available to the network, in a similar fashion as our previous study connecting MRI signals and histology (Liang et al 2022).…”

“…In comparison, the deep learning framework is data-driven and model-free, and there have been several reports on using machine and deep learning to predict MRI-based tissue parameters, including FODs, from MRI signals (Lin et al 2019; Gibbons et al 2019; Li et al 2021). Our recent study has demonstrated that the deep learning framework can be used to generate virtual histology from MRI signals by training a network with co-registered MRI and histological data (Liang et al 2022). The data from the Allen Mouse Brain Connectivity Atlas (AMBCA)(Oh et al 2014; Kuan et al 2015; Lein et al 2007), which contains an immense database of viral tracer data of the mouse brain captured using serial two-photon microscopy, provide rich information on mouse brain structural connectivity at the mesoscopic level.…”

“…There have been several reports on using machine and deep learning to predict MRI-based tissue parameters, including FODs, from MRI signals (Gibbons et al, 2019; Li et al, 2021; Lin et al, 2019), bypassing time-consuming model fitting procedures. Our recent study also demonstrated that the deep learning framework can be used to generate virtual histology from MRI signals by training a network with co-registered MRI and histological data (Liang et al, 2022). With the data from the Allen Mouse Brain Connectivity Atlas (AMBCA) (Kuan et al, 2015; Lein et al, 2007; Oh et al, 2014), which contains a large collection of mouse brain tract-tracing data captured using serial two-photon microscopy, we can now access rich information on mouse brain structural connectivity at the mesoscopic level and use it to guide the estimation of FODs from dMRI signals.…”