Fast and robust multimodal image registration using a local derivative pattern

Abstract

Purpose: Deformable multimodal image registration, which can benefit radiotherapy and image guided surgery by providing complementary information, remains a challenging task in the medical image analysis field due to the difficulty of defining a proper similarity measure. This article presents a novel, robust and fast binary descriptor, the discriminative local derivative pattern (dLDP), which is able to encode images of different modalities into similar image representations. Methods: dLDP calculates a binary string for each voxel according to the pattern of intensity derivatives in its neighborhood. The descriptor similarity is evaluated using the Hamming distance, which can be efficiently computed, instead of conventional L1 or L2 norms. For the first time, we validated the effectiveness and feasibility of the local derivative pattern for multimodal deformable image registration with several multi-modal registration applications. Results: dLDP was compared with three state-of-the-art methods in artificial image and clinical settings. In the experiments of deformable registration between different magnetic resonance imaging (MRI) modalities from BrainWeb, between computed tomography and MRI images from patient data, and between MRI and ultrasound images from BITE database, we show our method outperforms localized mutual information and entropy images in terms of both accuracy and time efficiency. We have further validated dLDP for the deformable registration of preoperative MRI and three-dimensional intraoperative ultrasound images. Our results indicate that dLDP reduces the average mean target registration error from 4.12 mm to 2.30 mm. This accuracy is statistically equivalent to the accuracy of the state-of-the-art methods in the study; however, in terms of computational complexity, our method significantly outperforms other methods and is even comparable to the sum of the absolute difference. Conclusions: The results reveal that dLDP can achieve superior performance regarding both accuracy and time efficiency in general multimodal image registration. In addition, dLDP also indicates the potential for clinical ultrasound guided intervention.

Algorithm Selection in Multimodal Medical Image Registration

“…The registration process by combining the structural images, such as CT or MRI images, with functional images, such as PET or SPECTis used in different applications such as disease diagnosis and computer-aided surgery [16][17][18][19]. Combined information from the variability of images, for instance, Computer tomography (CT), is applied to obtain more comprehensive data about the patient.…”

Radiation Oncology in the Era of Big Data and Machine Learning for Precision Medicine

“…Various machine learning-based methods [41][42][43][44][45][46][47] for image registration have proposed by investigators to not only align the anatomical structures but also alleviate the appearance difference. Hu et al [41] proposed a method based on regression forest for image registration of two arbitrary MR images.…”

“…The results showed that such an approach like CNN-based model can significantly outperform other state-of-the-art methods. Jiang et al [43] employed a discriminative local derivative pattern method to achieve fast and robust multimodal image registration. The results revealed that the proposed method can achieve superior performance regarding accuracy in multimodal image registration as well as also indicated the potential for clinical US-guided intervention.…”

Deformable multi-modal registration using 3D-FAST conditioned mutual information

“…In the FastMI registration, 3 D-FAST was performed using 90 neighborhoods with a 3 Â 3 Gaussian filter mask and a threshold of M ¼ 0.1 in Equation (10). The window's width of structural distance q u in Equation (13) is set to be 20.…”

“…For example, Wachinger et al [7] transformed multi-modal images into entropy or Laplacian images. Heinrich et al [8,9] and Jiang et al [10,11] transferred multi-modal images to unimodal images with local structure descriptor named 'Modality Independent Neighborhood Descriptor' (MIND), 'Self-Similarity Context' (SSC), 'discriminative Local Derivative Pattern' (dLDP) and 'modality independent Local Binary Pattern' (miLBP). Although methods in this category have higher registration speed, they may not be applicable for some tissues or organs with less structural information and the modality transformation may lose a large amount of image information.…”