Ev-ReconNet: Visual Place Recognition Using Event Camera With Spiking Neural Networks

Abstract

In this paper, we utilize the advantages of an event camera to tackle the visual place recognition (VPR) problem. The event camera's high measurement rate, low latency, and high dynamic range make it well-suited to overcome the limitations of conventional vision sensors. However, to apply the existing convolutional neural networks (CNNs) based algorithms such as NetVLAD, the asynchronous event stream should be converted to a synchronous image frame, which causes a loss in temporal information. To address this problem, this paper proposes a method that employs the asynchronous characteristic of spiking neural networks (SNNs) to leverage the temporal nature of event streams. The event stream is converted to event images and tensors in our preprocessing module. The SNNs-based reconstruction networks, which are converted from CNNs, reconstruct edge images from event tensors regardless of external environment changes. Visual place recognition is conducted by matching features of the database and those from NetVLAD, which we used as a feature extraction network in this study. To evaluate the performance of VPR by comparing the previous methods for DDD17 and the Brisbane-Event-VPR dataset. Experimental results demonstrate that the matching accuracy of the proposed method is better than previous methods, especially for datasets with adverse weather conditions. We also verify that the performance and energy efficiency are improved with SNNs over CNNs. Our code is available for download on https://github.com/AIRLABkhu/EvReconNet.

Artificial intelligence-based spatio-temporal vision sensors: applications and prospects

“…Valeiras et al (Valeiras and Clady, 2018) used iterative optimization and least squares fitting method to detect the straight line of the pulsed flow. In addition, Lee et al (Lee and Hwang, 2023) defined a buffer of pulse streams for object edge detection.…”