Semi-supervised Keypoint Detector and Descriptor for Retinal Image Matching

Liu, Jiazhen; Li, Xirong; Wei, Qijie; Xu, Jie; Ding, Dayong

doi:10.1007/978-3-031-19803-8_35

Cited by 13 publications

(5 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To demonstrate the registration performance of our work, we compare it with both traditional methods (SIFT (Lowe 2004), PBO (Oinonen et al 2010), REMPE (Hernandez-Matas, Zabulis, and Argyros 2020)) and deep learningbased methods (SuperPoint (DeTone, Malisiewicz, and Rabinovich 2018), GLAMpoints (Truong et al 2019), R2D2 (Revaud et al 2019), SuperGlue (Sarlin et al 2020), NCNet (Rocco et al 2020), SuperRetina (Liu et al 2022)). Table 1 summarizes the comparison among these methods.…”

Section: Comparison With Other Methodsmentioning

confidence: 99%

“…To match the keypoints based on descriptors, we first train a SuperGlue (Sarlin et al 2020) model based on optimal transport. Then a mapping between the two images is computed using homography given matched points, following many earlier methods (DeTone, Malisiewicz, and Rabinovich 2018; Sarlin et al 2020;Liu et al 2022). Homography estimation is a good solution for linear registration tasks as it reduces errors in keypoint detection and matching.…”

Section: Retinal Image Matchingmentioning

confidence: 99%

“…Image registration is to spatially align two or more images into the same coordinate system. It has received much attention in the past years as it is often important to align multiple retinal images for ocular disease diagnosis and management (Lee et al 2019;Liu et al 2022;Motta, Casaca, and Paiva 2019;Ding et al 2023). There are many challenges in retinal image registration.…”

Section: Introductionmentioning

confidence: 99%

“…Moreover, the random sampling of negative points is ineffective. Depart from SuperPoint, Liu et al (Liu et al 2022) propose SuperRetina, which is an endto-end method for retinal image matching with jointly trainable keypoint detector and descriptor. However, additional manually annotated data are used.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

SuperJunction: Learning-Based Junction Detection for Retinal Image Registration

Wang,

et al. 2024

AAAI

View full text Add to dashboard Cite

Keypoints-based approaches have shown to be promising for retinal image registration, which superimpose two or more images from different views based on keypoint detection and description. However, existing approaches suffer from ineffective keypoint detector and descriptor training. Meanwhile, the non-linear mapping from 3D retinal structure to 2D images is often neglected. In this paper, we propose a novel learning-based junction detection approach for retinal image registration, which enhances both the keypoint detector and descriptor training. To improve the keypoint detection, it uses a multi-task vessel detection to regularize the model training, which helps to learn more representative features and reduce the risk of over-fitting. To achieve effective training for keypoints description, a new constrained negative sampling approach is proposed to compute the descriptor loss. Moreover, we also consider the non-linearity between retinal images from different views during matching. Experimental results on FIRE dataset show that our method achieves mean area under curve of 0.850, which is 12.6% higher than 0.755 by the state-of-the-art method. All the codes are available at https://github.com/samjcheng/SuperJunction.

show abstract

Section: Comparison With Other Methodsmentioning

confidence: 99%

Section: Retinal Image Matchingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

SuperJunction: Learning-Based Junction Detection for Retinal Image Registration

Wang,

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…We compare with the following five domain-specific methods: REMPE [42], HM-16 [43], Harris-PIIFD [44], GDB-ICP [45], and SuperRetina [46].…”

Section: Experiments Setupmentioning

confidence: 99%

INSPIRE: Intensity and spatial information-based deformable image registration

2023

View full text Add to dashboard Cite

We present INSPIRE, a top-performing general-purpose method for deformable image registration. INSPIRE brings distance measures which combine intensity and spatial information into an elastic B-splines-based transformation model and incorporates an inverse inconsistency penalization supporting symmetric registration performance. We introduce several theoretical and algorithmic solutions which provide high computational efficiency and thereby applicability of the proposed framework in a wide range of real scenarios. We show that INSPIRE delivers highly accurate, as well as stable and robust registration results. We evaluate the method on a 2D dataset created from retinal images, characterized by presence of networks of thin structures. Here INSPIRE exhibits excellent performance, substantially outperforming the widely used reference methods. We also evaluate INSPIRE on the Fundus Image Registration Dataset (FIRE), which consists of 134 pairs of separately acquired retinal images. INSPIRE exhibits excellent performance on the FIRE dataset, substantially outperforming several domain-specific methods. We also evaluate the method on four benchmark datasets of 3D magnetic resonance images of brains, for a total of 2088 pairwise registrations. A comparison with 17 other state-of-the-art methods reveals that INSPIRE provides the best overall performance. Code is available at github.com/MIDA-group/inspire.

show abstract

ConKeD: multiview contrastive descriptor learning for keypoint-based retinal image registration

Rivas-Villar,

Hervella,

Rouco

et al. 2024

Med Biol Eng Comput

View full text Add to dashboard Cite

Retinal image registration is of utmost importance due to its wide applications in medical practice. In this context, we propose ConKeD, a novel deep learning approach to learn descriptors for retinal image registration. In contrast to current registration methods, our approach employs a novel multi-positive multi-negative contrastive learning strategy that enables the utilization of additional information from the available training samples. This makes it possible to learn high-quality descriptors from limited training data. To train and evaluate ConKeD, we combine these descriptors with domain-specific keypoints, particularly blood vessel bifurcations and crossovers, that are detected using a deep neural network. Our experimental results demonstrate the benefits of the novel multi-positive multi-negative strategy, as it outperforms the widely used triplet loss technique (single-positive and single-negative) as well as the single-positive multi-negative alternative. Additionally, the combination of ConKeD with the domain-specific keypoints produces comparable results to the state-of-the-art methods for retinal image registration, while offering important advantages such as avoiding pre-processing, utilizing fewer training samples, and requiring fewer detected keypoints, among others. Therefore, ConKeD shows a promising potential towards facilitating the development and application of deep learning-based methods for retinal image registration. Graphical abstract

show abstract

Semi-supervised Keypoint Detector and Descriptor for Retinal Image Matching

Cited by 13 publications

References 30 publications

SuperJunction: Learning-Based Junction Detection for Retinal Image Registration

SuperJunction: Learning-Based Junction Detection for Retinal Image Registration

INSPIRE: Intensity and spatial information-based deformable image registration

ConKeD: multiview contrastive descriptor learning for keypoint-based retinal image registration

Contact Info

Product

Resources

About