Deep-Learning Image Stabilization for Adaptive Optics Ophthalmoscopy

Abstract

An adaptive optics scanning laser ophthalmoscope (AOSLO) has the characteristics of a high resolution and a small field of view (FOV), which are greatly affected by eye motion. Continual eye motion will cause distortions both within the frame (intra-frame) and between frames (inter-frame). Overcoming eye motion and achieving image stabilization is the first step and is of great importance in image analysis. Although cross-correlation-based methods enable image registration to be achieved, the manual identification and distinguishing of images with saccades is required; manual registration has a high accuracy, but it is time-consuming and complicated. Some imaging systems are able to compensate for eye motion during the imaging process, but special hardware devices need to be integrated into the system. In this paper, we proposed a deep-learning-based algorithm for automatic image stabilization. The algorithm used the VGG-16 network to extract convolution features and a correlation filter to detect the position of reference in the next frame, and finally, it compensated for displacement to achieve registration. According to the results, the mean difference in the vertical and horizontal displacement between the algorithm and manual registration was 0.07 pixels and 0.16 pixels, respectively, with a 95% confidence interval of (−3.26 px, 3.40 px) and (−4.99 px, 5.30 px). The Pearson correlation coefficients for the vertical and horizontal displacements between these two methods were 0.99 and 0.99, respectively. Compared with cross-correlation-based methods, the algorithm had a higher accuracy, automatically removed images with blinks, and corrected images with saccades. Compared with manual registration, the algorithm enabled manual registration accuracy to be achieved without manual intervention.

All‐Optically Controlled Retinomorphic Memristor for Image Processing and Stabilization

“…Thus, image stabilization technology is an indispensable component of artificial visual systems. [8,9] Traditional electronic image stabilization (EIS) technology utilizes image processing methods to compensate for interframe jitter; [10,11] however these approaches require significant computing and storage resources. Moreover, the physical separation of the image sensing, memory, and processing modules results in high energy expenditure and a significant latency time.…”