Accurate and Robust Alignment of Differently Stained Histologic Images Based on Greedy Diffeomorphic Registration

Abstract

Histopathologic assessment routinely provides rich microscopic information about tissue structure and disease process. However, the sections used are very thin, and essentially capture only 2D representations of a certain tissue sample. Accurate and robust alignment of sequentially cut 2D slices should contribute to more comprehensive assessment accounting for surrounding 3D information. Towards this end, we here propose a two-step diffeomorphic registration approach that aligns differently stained histology slides to each other, starting with an initial affine step followed by estimating a deformation field. It was quantitatively evaluated on ample (n = 481) and diverse data from the automatic non-rigid histological image registration challenge, where it was awarded the second rank. The obtained results demonstrate the ability of the proposed approach to robustly (average robustness = 0.9898) and accurately (average relative target registration error = 0.2%) align differently stained histology slices of various anatomical sites while maintaining reasonable computational efficiency (<1 min per registration). The method was developed by adapting a general-purpose registration algorithm designed for 3D radiographic scans and achieved consistently accurate results for aligning high-resolution 2D histologic images. Accurate alignment of histologic images can contribute to a better understanding of the spatial arrangement and growth patterns of cells, vessels, matrix, nerves, and immune cell interactions.

DeepHistReg: Unsupervised Deep Learning Registration Framework for Differently Stained Histology Samples

“…The method was the winner of the ANHIR challenge and compared to other classical registration approaches was amazingly fast due to a well-optimized, commercial implementation. Other methods dedicated to histology registration were introduced by researchers from the University of Pennsylvania (UPENN) [15] and the AGH University (AGH) [16]. These teams achieved the second and third best scores in the ANHIR challenge.…”

“…This step could be done differently (e.g. by color deconvolution as in [15]) but we decided that deep segmentation is fast, robust, and easily convertible to other histology datasets. The visualization of an example source-target pair after the preprocessing is shown in Figure 1b.…”

Learning-Based Affine Registration of Histological Images

“…One of the most difficult subproblems for the challenge participants was to calculate the initial, global transform. It was a key to success and all the best scoring teams put a significant effort to do this correctly, resulting in algorithms based on combined brute force and iterative alignment [8,9], or applying a fixed number of random transformations [10]. In this work, we propose a method based on deep learning which makes the process significantly faster, more robust, without the necessity to manually find a set of parameters viable for all the image pairs.…”

“…The main advantage of the learning-based approach over the classical, iterative optimization is a fast, usually real-time registration, which makes the algorithms more useful in clinical practice. During the ANHIR challenge the best scoring teams [8][9][10] used the classical approach. However, we think that it is reasonable to solve the problem using deep networks, potentially both improving the results and decreasing the computation time.…”

Unsupervised Learning-Based Nonrigid Registration of High Resolution Histology Images

“…The researchers used the normalized gradient field (NGF) similarity metric [9] and strongly optimized code resulting in undoubtedly the best and clinically applicable method. The team with the second-best score (UPENN) [10] proposed an algorithm consisting of background removal by stain deconvolution, random initial alignment, affine registration and diffeomorphic, nonrigid registration based on the Greedy tool [11,12]. The third best team (AGH) [13] developed a method similar to the winners with the differences that instead of the NGF they used the modality independent neighborhood descriptor (MIND) [14], and used the Demons algorithm to directly optimize the dense deformation field replacing the B-Spline deformation model, as in [8].…”