Scalable High-Performance Image Registration Framework by Unsupervised Deep Feature Representations Learning

Abstract

Feature selection is a critical step in deformable image registration. In particular, selecting the most discriminative features that accurately and concisely describe complex morphological patterns in image patches improves correspondence detection, which in turn improves image registration accuracy. Furthermore, since more and more imaging modalities are being invented to better identify morphological changes in medical imaging data,, the development of deformable image registration method that scales well to new image modalities or new image applications with little to no human intervention would have a significant impact on the medical image analysis community. To address these concerns, a learning-based image registration framework is proposed that uses deep learning to discover compact and highly discriminative features upon observed imaging data. Specifically, the proposed feature selection method uses a convolutional stacked auto-encoder to identify intrinsic deep feature representations in image patches. Since deep learning is an unsupervised learning method, no ground truth label knowledge is required. This makes the proposed feature selection method more flexible to new imaging modalities since feature representations can be directly learned from the observed imaging data in a very short amount of time. Using the LONI and ADNI imaging datasets, image registration performance was compared to two existing state-of-the-art deformable image registration methods that use handcrafted features. To demonstrate the scalability of the proposed image registration framework image registration experiments were conducted on 7.0-tesla brain MR images. In all experiments, the results showed the new image registration framework consistently demonstrated more accurate registration results when compared to state-of-the-art.

Deep Learning in Medical Image Analysis

“…, not guaranteed to work for other image types. For instance, the methods of image segmentation and registration designed for 1.5-Tesla T1-weighted brain MR images are not applicable to 7.0-Tesla T1-weighted MR images (48, 24), not to mention to other modalities or different organs. Further, as demonstrated in (78), 7.0-Tesla MR images can reveal the brain’s anatomy with the resolution equivalent to that obtained from thin slices in vitro.…”

“…Thanks to its nice characteristic of learning hierarchical feature representations solely from data, deep learning has achieved record-breaking performance in a variety of artificial intelligence applications (11, 12, 13, 14, 15, 16, 17, 18) and grand challenges (19, 20, 21). Particularly, great improvements in computer vision inspired its use to medical image analysis such as image segmentation (22, 23), image registration (24), image fusion (25), image annotation (26), computer-aided diagnosis and prognosis (27, 28, 29), lesion/landmark detection (30, 31, 32), and microscopic imaging analysis (33, 34), to name a few.…”

“…, the structural or configural information among neighboring pixels or voxels is another important source of information), convolutional neural networks have gained great interest in medical image analysis (33, 45, 32, 46, 22, 44, 31). However, networks with vectorized inputs were also successfully applied to different medical applications (47, 25, 27, 29, 48, 24, 49, 50). Along with deep neural networks, deep generative models (51) such as deep belief networks and deep Boltzmann machines that are the probabilistic graphical models with multiple layers of hidden variables have also been successfully applied to brain disease diagnosis (43, 25, 52, 29), lesion segmentation (53, 45, 32, 54), cell segmentation (33, 55, 34, 56), image parsing (57, 58, 59), and tissue classification (31, 46, 22, 44).…”

ssEMnet: Serial-Section Electron Microscopy Image Registration Using a Spatial Transformer Network with Learned Features

“…Convolutional neural networks (CNNs) and their variants have shown unprecedented potential by largely outperforming conventional computer vision algorithms using hand-crafted feature descriptors, but their application to ssEM image registration has not been explored. Wu et al [10] used a 3D autoencoder to extract features from MRI volumes, which are then combined with a conventional sparse, feature-driven registration method. Recent work by Jaderberg et al [6] on the spatial transformer network (STN) uses a differentiable network module inside a CNN to overcome the drawbacks of CNNs (i.e., lack of scale-and rotationinvariance).…”

Computational medicine: A cybernetic eye for rare disease

“…In medical-image analysis more generally, deep learning is becoming a widely used tool for categorizing medical images 8,9 . Typical applications include medical-image enhancement 10 , registration 11 , segmentation 12,13 and high-level understanding for diagnosis or clinical evaluation, for example in Alzheimer’s disease 14 and cancers 15 . By pushing CNN technology into clinical settings, and showing that it can perform well for a rare disease, Liu and colleagues’ multihospital collaborative AI platform represents a step towards improving the medical management of rare diseases, in particular in under-resourced areas.…”