SiamEvent: Event-based Object Tracking via Edge-aware Similarity Learning with Siamese Networks

Abstract

Event cameras are novel sensors that perceive the per-pixel intensity changes and output asynchronous event streams, showing lots of advantages over traditional cameras, such as high dynamic range (HDR) and no motion blur. It has been shown that events alone can be used for object tracking by motion compensation or prediction. However, existing methods assume that the target always moves and is the stand-alone object. Moreover, they fail to track the stopped nonindependent moving objects on fixed scenes. In this paper, we propose a novel event-based object tracking framework, called SiamEvent, using Siamese networks via edge-aware similarity learning. Importantly, to find the part having the most similar edge structure of target, we propose to correlate the embedded events at two timestamps to compute the target edge similarity. The Siamese network enables tracking arbitrary target edge by finding the part with the highest similarity score. This extends the possibility of event-based object tracking applied not only for the independent stand-alone moving objects, but also for various settings of the camera and scenes. In addition, target edge initialization and edge detector are also proposed to prevent SiamEvent from the drifting problem. Lastly, we built an open dataset including various synthetic and real scenes to train and evaluate SiamEvent. The dataset is available at https: //github.com/yujeong-star/SiamEvent. Extensive experiments demonstrate that SiamEvent achieves up to 15% tracking performance enhancement than the baselines on the real-world scenes and more robust tracking performance in the challenging HDR and motion blur conditions.

Deep Learning for HDR Imaging: State-of-the-Art and Future Trends

“…Event cameras sample light based on scene dynamics rather than a fixed frame rate, which is unrelated to the viewed scenes. Neuromorphic cameras are different in principle from conventional cameras [124], [125]. Unlike conventional cameras that capture intensity values at a fixed frame rate, event cameras sense changes in intensity asynchronously at the time they occur.…”