Learning Long-Horizon Robot Exploration Strategies for Multi-Object Search in Continuous Action Spaces

Fabian, Schmalstieg,; Honerkamp, Daniel; Welschehold, Tim; Valada, Abhinav

doi:10.48550/arxiv.2205.11384

Cited by 2 publications

(3 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Wixson and Ballard [28] remarks that selecting views for object search in a local region is a harder problem than the selection of which region to search in. Most works that demonstrate robotic search within a search region reduce the problem to 2D [29,6,30,7,31,32,33,34]. For a literature survey and taxonomy of object search, refer to [12].…”

Section: Related Workmentioning

confidence: 99%

“…Deep learning methods that typically map raw observations to actions [35,36,37,38,34] can enable 3D object search, yet it is hard to train such a model on a robot and ensure generalization to a new real-world environment; ongoing work (e.g., [34]) is addressing this challenge. In contrast, GenMOS only requires basic perception capabilities such as object detection and localization to enable object search.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A System for Generalized 3D Multi-Object Search

Zheng¹,

Anirudha²,

Tellex³

2023

Preprint

View full text Add to dashboard Cite

Searching for objects is a fundamental skill for robots. As such, we expect object search to eventually become an off-the-shelf capability for robots, similar to e.g., object detection and SLAM. In contrast, however, no system for 3D object search exists that generalizes across real robots and environments. In this paper, building upon a recent theoretical framework that exploited the octree structure for representing belief in 3D, we present GenMOS (Generalized Multi-Object Search), the first general-purpose system for multiobject search (MOS) in a 3D region that is robot-independent and environment-agnostic. GenMOS takes as input point cloud observations of the local region, object detection results, and localization of the robot's view pose, and outputs a 6D viewpoint to move to through online planning. In particular, GenMOS uses point cloud observations in three ways: (1) to simulate occlusion; (2) to inform occupancy and initialize octree belief; and (3) to sample a belief-dependent graph of view positions that avoid obstacles. We evaluate our system both in simulation and on two real robot platforms. Our system enables, for example, a Boston Dynamics Spot robot to find a toy cat hidden underneath a couch in under one minute. We further integrate 3D local search with 2D global search to handle larger areas, demonstrating the resulting system in a 25m 2 lobby area.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

A System for Generalized 3D Multi-Object Search

Zheng¹,

Anirudha²,

Tellex³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…However, they are computationally expensive, often do not optimize past a limited horizon, and can struggle when objectives oppose each other. Learning-based methods efficiently learn directly from high-dimensional observations and are well suited to handle unexplored environments [14]- [16]. Nevertheless, they either restrict the action space [14], [17] or rely on expert demonstrations [15] to cope with the highdimensional action space and long-horizon nature of mobile manipulation.…”

Section: Introductionmentioning

confidence: 99%

N$^2$M$^2$: Learning Navigation for Arbitrary Mobile Manipulation Motions in Unseen and Dynamic Environments

Honerkamp¹,

Welschehold²,

Valada³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Despite its importance in both industrial and service robotics, mobile manipulation remains a significant challenge as it requires a seamless integration of end-effector trajectory generation with navigation skills as well as reasoning over long-horizons. Existing methods struggle to control the large configuration space, and to navigate dynamic and unknown environments. In previous work, we proposed to decompose mobile manipulation tasks into a simplified motion generator for the end-effector in task space and a trained reinforcement learning agent for the mobile base to account for kinematic feasibility of the motion. In this work, we introduce Neural Navigation for Mobile Manipulation (N 2 M 2 ) which extends this decomposition to complex obstacle environments and enables it to tackle a broad range of tasks in real world settings. The resulting approach can perform unseen, long-horizon tasks in unexplored environments while instantly reacting to dynamic obstacles and environmental changes. At the same time, it provides a simple way to define new mobile manipulation tasks. We demonstrate the capabilities of our proposed approach in extensive simulation and real-world experiments on multiple kinematically diverse mobile manipulators. Code and videos are publicly available at http://mobile-rl.cs.uni-freiburg.de.

show abstract

Learning Long-Horizon Robot Exploration Strategies for Multi-Object Search in Continuous Action Spaces

Cited by 2 publications

References 19 publications

A System for Generalized 3D Multi-Object Search

A System for Generalized 3D Multi-Object Search

N$^2$M$^2$: Learning Navigation for Arbitrary Mobile Manipulation Motions in Unseen and Dynamic Environments

Contact Info

Product

Resources

About