VoxelMorph: A Learning Framework for Deformable Medical Image Registration

Abstract

We present VoxelMorph, a fast learning-based framework for deformable, pairwise medical image registration. Traditional registration methods optimize an objective function for each pair of images, which can be time-consuming for large datasets or rich deformation models. In contrast to this approach, and building on recent learning-based methods, we formulate registration as a function that maps an input image pair to a deformation field that aligns these images. We parameterize the function via a convolutional neural network (CNN), and optimize the parameters of the neural network on a set of images. Given a new pair of scans, VoxelMorph rapidly computes a deformation field by directly evaluating the function. In this work, we explore two different training strategies. In the first (unsupervised) setting, we train the model to maximize standard image matching objective functions that are based on the image intensities. In the second setting, we leverage auxiliary segmentations available in the training data. We demonstrate that the unsupervised model's accuracy is comparable to state-of-the-art methods, while operating orders of magnitude faster. We also show that VoxelMorph trained with auxiliary data improves registration accuracy at test time, and evaluate the effect of training set size on registration. Our method promises to speed up medical image analysis and processing pipelines, while facilitating novel directions in learning-based registration and its applications. Our code is freely available at http://voxelmorph.csail.mit.edu.

Automatic Landmark-guided Bijective Brain Image Registration by Composing Region-based Locally Diffeomorphic Warpings

“…Klein et al evaluated 14 methods with four MRI dataset of healthy and young subjects, and showed that these methods perform well in warping the sub-cortical regions (average DSC above 80%), but even the top performing method ANTS [5] generally has poor performance in warping cortical regions (average DSC between 60% and 70%) [13]. The recently-developed deep learning methods still have comparable performance as ANTS [6], [7]. It is because almost all of the widely used brain image registration techniques that work in the 3D Euclidean space, whether volume-based, or surface and volume hybrid methods, are based on solving the optimization problem of matching the whole brain at once and suffer from the local minimum problem, resulting in poor registration of brain cortical regions.…”

“…For quantitative evaluation, we used the global warping fields obtained above to warp each subject's FreeSurfer delineated regions, described in Section II, onto MNI152 space (cortical regions include banks of superior temporal sulcus, caudal anterior cingulate, caudal middle frontal, corpus callosum, cuneus, entorhinal, fusiform, inferior parietal, inferior temporal, isthmus cingulate, lateral occipital, lateral orbitofrontal, lingual, medial orbitofrontal, middle temporal, parahippocampal, paracentral, pars opercularis, pars orbitalis, pars triangularis, pericalcarine, postcentral, posterior cingulate, precentral, precuneus, rostral anterior cingulate, rostral middle frontal, superior frontal, superior parietal, superior temporal, supramarginal, frontal pole, temporal pole, transverse temporal and insula). As done previously [6], [7], [13], we have evaluated our registration accuracy using the DSC between the regional binary masks of the warped and corresponding target FreeSurfer regions. Figure 6 illustrates the distribution of the DSC between corresponding regions for all cortical regions in FreeSurfer using boxplot for both LG-RBSN (in blue) and ANTS (in red) categorized in five lobar brain segments.…”

“…The most commonly employed spatial normalization methods perform either a volume-based non-linear registration of structural images in 3D Euclidean space, or a surface-based non-linear registration of the cerebral cortex surfaces in 2D parametric surface space. For example, currently widely-used volume-based brain image registration methods include large deformation diffeomorphic metric mapping (LDDMM) [4], advanced normalization tools (ANTS) [5], Quicksilver [6], and VoxelMorph [7]. Alternatively, commonly used surface-based methods include FreeSurfer [8] and spherical demons [9].…”

Quantitative brain relaxation atlases for personalized detection and characterization of brain pathology

“…Elastix was used for performance reasons because pair‐wise registrations with ANTs typically require a computational time ranging from 30 min to 1 h, whereas Elastix requires around 3 min. Novel methods based on deep learning approaches may be investigated in the future to replace the current registration tools.…”

Two‐dimensional ultrasound‐computed tomography image registration for monitoring percutaneous hepatic intervention

“…46 Recent advancements in computing and automatic image analysis have been able to significantly reduce the extra time required for the image registration so that image fusion becomes a feasible option in ablation procedures. [47][48][49][50][51] This initial rigid registration can be achieved by defining a minimum of three (3) noncollinear common fiducial points in both datasets, or by defining a common plane and single point. 52 Classical methods for implementing multi-modality image registration use mutual information to measure differences and a rigid transformation to warp the previously acquired data with the real-time US B-mode image.…”