Adaptive fovea for scanning depth sensors

Abstract

Depth sensors have been used extensively for perception in robotics. Typically these sensors have a fixed angular resolution and field of view (FOV). This is in contrast to human perception, which involves foveating: scanning with the eyes’ highest angular resolution over regions of interest (ROIs). We build a scanning depth sensor that can control its angular resolution over the FOV. This opens up new directions for robotics research, because many algorithms in localization, mapping, exploration, and manipulation make implicit assumptions about the fixed resolution of a depth sensor, impacting latency, energy efficiency, and accuracy. Our algorithms increase resolution in ROIs either through deconvolutions or intelligent sample distribution across the FOV. The areas of high resolution in the sensor FOV act as artificial fovea and we adaptively vary the fovea locations to maximize a well-known information theoretic measure. We demonstrate novel applications such as adaptive time-of-flight (TOF) sensing, LiDAR zoom, gradient-based LiDAR sensing, and energy-efficient LiDAR scanning. As a proof of concept, we mount the sensor on a ground robot platform, showing how to reduce robot motion to obtain a desired scanning resolution. We also present a ROS wrapper for active simulation for our novel sensor in Gazebo. Finally, we provide extensive empirical analysis of all our algorithms, demonstrating trade-offs between time, resolution and stand-off distance.

Robotics Perception and Control: Key Technologies and Applications

“…Despite its promising capabilities, the method's robustness across diverse conditions remains to be thoroughly validated, thus pointing toward a need for expanded testing and refinement in self-assessment techniques. Tasneem et al [76] introduced an adaptive foveation method for scanning depth sensors, thus enabling the dynamic focusing of sensor resolution on areas of interest within its field of view. This approach, through strategic resolution allocation and deconvolution, allows for the creation of high-resolution "artificial foveae" that adapt to maximize data collection efficiency.…”

Design of an Adaptive Lightweight LiDAR to Decouple Robot-Camera Geometry

“…More generally, such compensation allows the robot to focus on the control task while the camera can perform perception (which is required for the control task) independently, and greatly simplifies robot planning as the planner does not need to account for perception and just needs to reason about the control task at hand. MEMS LiDAR optics have the advantages of small size and low power consumption [44,23,24]. Our algorithmic and system design contributions beyond this are:…”

“…Small, compact LiDAR for small robotics: MEMS mirrors have been studied to build compact LiDAR systems [44,23,24]. For instance, Kasturi et al demonstrated a UVA-borne LiDAR with an electrostatic MEMS mirror scanner that could fit into a small volume of 70 mm × 60 mm × 60 mm and weighed about only 50 g [23].…”

Fast Task-Based Adaptive Sampling for 3D Single-Photon Multispectral Lidar Data

“…For instance, a data set with 100 × 100 pixels, 1000 time bins, and 32 wavelengths can yield an excessively large number of data samples ( >10 8 ) which will lead to a significant computational load and prohibitive memory requirements. Adaptive sampling appears as a promising strategy to reduce data volume [17], [20], [25], [26], [31] by focusing the scanning on pixels containing target's returns, and scanning less those only containing background reflections.…”

“…We distinguish two main approaches to improve data acquisition using subsampling, those considering spatially structured scanned points, and those based on random points. Structured point scanning includes foveated based scanning, which mimic the vision system found in the animal kingdom [25]- [27]. Such systems consider a structured scanning array (e.g., a circle) which allocates denser points in regions of interest and sparser points in the remaining regions.…”