Recurrent 3D attentional networks for end-to-end active object recognition

Abstract

Active vision is inherently attention-driven: The agent actively selects views to attend in order to fast achieve the vision task while improving its internal representation of the scene being observed. Inspired by the recent success of attention-based models in 2D vision tasks based on single RGB images, we propose to address the multi-view depthbased active object recognition using attention mechanism, through developing an end-to-end recurrent 3D attentional network. The architecture takes advantage of a recurrent neural network (RNN) to store and update an internal representation. Our model, trained with 3D shape datasets, is able to iteratively attend to the best views targeting an object of interest for recognizing it. To realize 3D view selection, we derive a 3D spatial transformer network which is differentiable for training with backpropagation, achieving much faster convergence than the reinforcement learning employed by most existing attention-based models. Experiments show that our method, with only depth input, achieves state-of-the-art next-best-view performance in time efficiency and recognition accuracy.

An Azimuth Aware Deep Reinforcement Learning Framework for Active SAR Target Recognition

“…Deep Reinforcement Learning (DRL), combining the powerful approximation ability of Deep Neural Network (DNN) and the excellent decision-making ability of Reinforcement Learning (RL), has made great progress in fields like robot control [9], adversarial games [10], and the foundation model training [11] with the in-depth research in recent years. In the context of the AcTR task, this learning paradigm also suits very well, which can be validated by the extensive studies and applications [12]- [13]. The features of the observed images can be effectively extracted through DNN to construct the state in the Markov decision process (MDP).…”

Active Scene Understanding via Online Semantic Reconstruction

“…Song et al [SZX15] propose an information‐theoretic approach based on 3D volumetric deep learning [WSK∗15]. When target objects are unknown , detection and recognition need to be solved simultaneously [LSZ∗19]. Ye et al [YLL∗18] propose navigation policy learning guided by active object detection and recognition.…”