Object Reconstruction Based on Attentive Recurrent Network from Single and Multiple Images

Abstract

The application of traditional 3D reconstruction methods such as structure-from-motion and simultaneous localization and mapping are typically limited by illumination conditions, surface textures, and wide baseline viewpoints in the field of robotics. To solve this problem, many researchers have applied learning-based methods with convolutional neural network architectures. However, simply utilizing convolutional neural networks without taking other measures into account is computationally intensive, and the results are not satisfying. In this study, to obtain the most informative images for reconstruction, we introduce a residual block to a 2D encoder for improved feature extraction, and propose an attentive latent unit that makes it possible to select the most informative image being fed into the network rather than choosing one at random. The recurrent visual attentive network is injected into the auto-encoder network using reinforcement learning. The recurrent visual attentive network pays more attention to useful images, and the agent will quickly predict the 3D volume. This model is evaluated based on both single- and multi-view reconstructions. The experiment results show that the recurrent visual attentive network increases prediction performance in a way that is superior to other alternative methods, and our model has desirable capacity for generalization.

NAVS: A Neural Attention-Based Visual SLAM for Autonomous Navigation in Unknown 3D Environments

“…They implemented goal-oriented navigation while retaining the semantic information in the images [43,44]. Meanwhile, they used topological representation of space or transformer-based spatial planning for navigating [45,46].The use of an attention network injected into the neural network enables a greater focus on useful images and rapid prediction of objects to augment the representations of the depth map in detail [47,48].…”

NAS: A Neural Attention-based SLAM for Autonomous Navigation in Unknown 3D Environments

“…They implemented goaloriented navigation while retaining the semantic information in the images [43,44]. Meanwhile, they used topological representation of space or transformer-based spatial planning for navigating [45,46].The use of an attention network injected into the neural network enables a greater focus on useful images and rapid prediction of objects to augment the representations of the depth map in detail [47,48].…”

Hierarchical Patch Aggregation Transformer For Motion Deblurring

“…Amidst the proliferation of large-scale datasets and the rapid evolution of convolutional neural networks (CNNs), methods based on learning [16], [17], [18], [19], [20], [21], [22], [23] have achieved remarkable accomplishments in the realm of image restoration. These methods exploit abundant image data to implicitly learn the mapping relationship between blurry and clear images.…”

“…Particularly, the combination of encoder-decoder structures with residual learning or generative adversarial networks [26], [27] has been attempted to enhance performance. Nevertheless, the design of encoder-decoder architectures [20] predominantly focuses on recovering image features layer by layer, but relying solely on rudimentary local feature extraction mechanisms falls short of comprehensively capturing global semantic information within images. This limitation hampers the model's ability to accurately comprehend the underlying semantic information of blurry images.…”