A Fully Unsupervised Deep Learning Framework for Non-Rigid Fundus Image Registration

Abstract

In ophthalmology, the registration problem consists of finding a geometric transformation that aligns a pair of images, supporting eye-care specialists who need to record and compare images of the same patient. Considering the registration methods for handling eye fundus images, the literature offers only a limited number of proposals based on deep learning (DL), whose implementations use the supervised learning paradigm to train a model. Additionally, ensuring high-quality registrations while still being flexible enough to tackle a broad range of fundus images is another drawback faced by most existing methods in the literature. Therefore, in this paper, we address the above-mentioned issues by introducing a new DL-based framework for eye fundus registration. Our methodology combines a U-shaped fully convolutional neural network with a spatial transformation learning scheme, where a reference-free similarity metric allows the registration without assuming any pre-annotated or artificially created data. Once trained, the model is able to accurately align pairs of images captured under several conditions, which include the presence of anatomical differences and low-quality photographs. Compared to other registration methods, our approach achieves better registration outcomes by just passing as input the desired pair of fundus images.

Discriminative-Region Multi-Label Classification of Ultra-Widefield Fundus Images

“…Recently, deep learning has significantly improved the interpretation of fundus images, including tasks like vessel segmentation [ 16 , 17 ], optic disc and fovea (the center of the macula) localization [ 18 , 19 ], and image registration [ 20 , 21 ]. For vessel segmentation, researchers in [ 16 ] noticed that existing UNet-based models [ 22 ] can lose information due to multiple pooling steps and insufficient processing of local context features.…”

“…They first locate the optic disc using Faster RCNN [ 23 ], and then use its position to find the fovea within a ring-shaped area. In one paper [ 20 ], the authors proposed an unsupervised approach for retina fundus registration by combining two deep learning-based networks. A U-shaped fully convolutional neural network [ 24 ] and a spatial-transformer-type network [ 25 ] work together: the former finds matching points in fundus images, and the latter uses these points for geometric bilinear interpolation.…”

Retinal Images Registration via Unsupervised Deep Learning

“…For the purpose of this article, we focused solely on the Dice coefficient metric to present the results. For a more comprehensive evaluation, we recommend referring to the accompanying dissertation [24] and the following citations: [25], [26]. These references provide extensive insights and discussions on the performance and effectiveness of our proposed methodology, considering various metrics and experimental setups.…”

“…The author's dissertation contains the following articles: [25] and [26]. Others contributions were also developed in parallel, related to the participation in an extension project called GECET -Girls in Engineering, Exact Sciences and Technology: [30] and the medical and social research involving the mapping of COVID-19 risk groups in Brazil: [31].…”