A Keypoint Detection and Description Network Based on the Vessel Structure for Multi-modal Retinal Image Registration

Sindel, Aline; Hohberger, Bettina; Dehcordi, Sebastian Fassihi; Mardin, Christian Y.; Laemmer, Robert; Maier, Andreas; Christlein, Vincent

doi:10.1007/978-3-658-36932-3_12

Cited by 6 publications

(11 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We employ and extend the fully-convolutional RetinaCraquelureNet [20,19] for our end-to-end pipeline (see Fig. 1).…”

Section: Multi-modal Retinal Keypoint Detection and Description Networkmentioning

confidence: 99%

“…We set the feature dimension of the description head to 256-D to reduce the parameters for end-to-end learning. We pretrain the network from scratch using multi-modal retinal image patches centered at supervised keypoint positions with a binary cross-entropy loss for keypoint detection and a cross-modal bidirectional quadruplet descriptor loss [20,19]. Then, we fit the network into our pipeline.…”

Section: Multi-modal Retinal Keypoint Detection and Description Networkmentioning

confidence: 99%

“…With these modifications the predictions of the detector are scaled to the range zero to one. Then, we apply non-maximum suppression (NMS) to the keypoint confidence heatmap and extract the top N max keypoints from the NMS heatmap [20,19]. This step is non-differentiable, therefore we apply a differentiable subpixel keypoint refinement (DKR) that allows the gradients to flow back to the small regions around the keypoints.…”

Section: Multi-modal Retinal Keypoint Detection and Description Networkmentioning

confidence: 99%

“…The retinal method DeepSPA [12] uses a convolutional neural network (CNN) to classify patches of vessel junctions based on a step pattern representation. The keypoint detection and description network RetinaCraquelureNet [19] is trained on small multi-modal retinal image patches centered at vessel bifurcations and uses mutual nearest neighbor matching and random sample consensus (RANSAC) [7] for homography estimation. In GLAMpoints [22], homography guided self-supervised learning is applied to train a UNet [17] for keypoint detection combined with RootSIFT [1] descriptors for retinal image data.…”

Section: Introductionmentioning

confidence: 99%

“…ing Network (KPVSA-Net). We employ prior knowledge by extracting deep features of the vessel structure using the keypoint detection and description network RetinaCraquelureNet [19]. In contrast to vessel segmentation based methods, we extract the features directly from multi-modal images to learn distinctive crossmodal descriptors.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Multi-modal Retinal Image Registration Using a Keypoint-Based Vessel Structure Aligning Network

Sindel¹,

Hohberger²,

Maier³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

In ophthalmological imaging, multiple imaging systems, such as color fundus, infrared, fluorescein angiography, optical coherence tomography (OCT) or OCT angiography, are often involved to make a diagnosis of retinal disease. Multi-modal retinal registration techniques can assist ophthalmologists by providing a pixel-based comparison of aligned vessel structures in images from different modalities or acquisition times. To this end, we propose an end-to-end trainable deep learning method for multi-modal retinal image registration. Our method extracts convolutional features from the vessel structure for keypoint detection and description and uses a graph neural network for feature matching. The keypoint detection and description network and graph neural network are jointly trained in a self-supervised manner using synthetic multi-modal image pairs and are guided by synthetically sampled ground truth homographies. Our method demonstrates higher registration accuracy as competing methods for our synthetic retinal dataset and generalizes well for our real macula dataset and a public fundus dataset.

show abstract

“…We employ and extend the fully-convolutional RetinaCraquelureNet [20,19] for our end-to-end pipeline (see Fig. 1).…”

Section: Multi-modal Retinal Keypoint Detection and Description Networkmentioning

confidence: 99%

Section: Multi-modal Retinal Keypoint Detection and Description Networkmentioning

confidence: 99%

Section: Multi-modal Retinal Keypoint Detection and Description Networkmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Multi-modal Retinal Image Registration Using a Keypoint-Based Vessel Structure Aligning Network

Sindel¹,

Hohberger²,

Maier³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Multi-modal Retinal Image Registration Using a Keypoint-Based Vessel Structure Aligning Network

Sindel

Hohberger

Maier

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

A Multi-modal Registration and Visualization Software Tool for Artworks Using CraquelureNet

Sindel

Maier

Christlein

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

A Keypoint Detection and Description Network Based on the Vessel Structure for Multi-modal Retinal Image Registration

Cited by 6 publications

References 10 publications

Multi-modal Retinal Image Registration Using a Keypoint-Based Vessel Structure Aligning Network

Multi-modal Retinal Image Registration Using a Keypoint-Based Vessel Structure Aligning Network

Multi-modal Retinal Image Registration Using a Keypoint-Based Vessel Structure Aligning Network

A Multi-modal Registration and Visualization Software Tool for Artworks Using CraquelureNet

Contact Info

Product

Resources

About