VOILA: Visual-Observation-Only Imitation Learning for Autonomous Navigation

Karnan, Haresh; Warnell, Garrett; Xiao, Xuesu; Stone, Peter

doi:10.48550/arxiv.2105.09371

Cited by 4 publications

(4 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…At the same time, the whole simulator is highly modular and customizable, and new APIs and functionalities can be added on top of the existing simulator core. Additionally, the simulator allows human experts to control agents directly, supporting synthetic dataset creation and the development of imitation learning approaches [26], [27]. These objectives are obtained through the following implementation choices:…”

Section: Midgard Simulatormentioning

confidence: 99%

MIDGARD: A Simulation Platform for Autonomous Navigation in Unstructured Environments

Vecchio¹,

Palazzo²,

Guastella³

et al. 2022

Preprint

View full text Add to dashboard Cite

We present MIDGARD, an open source simulation platform for autonomous robot navigation in unstructured outdoor environments. We specifically design MIDGARD to enable training of autonomous agents (e.g., unmanned ground vehicles) in photorealistic 3D environments, and to support the generalization skills of learning-based agents by means of diverse and variable training scenarios. MIDGARD differs from other major simulation platforms in that it proposes a highly configurable procedural landscape generation pipeline, which enables autonomous agents to be trained in diverse scenarios while reducing the efforts and costs needed to create digital content from scratch.

show abstract

Section: Midgard Simulatormentioning

confidence: 99%

MIDGARD: A Simulation Platform for Autonomous Navigation in Unstructured Environments

Vecchio¹,

Palazzo²,

Guastella³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Robotic Teleoperation for Mobile Manipulation: IL leverages human demonstrations to learn tasks such as stationary manipulation [10,11,12,13] and navigation [14,15,16,17]. Having human operators remotely control an agent, or teleoperation, is a common approach for collecting demonstrations.…”

Section: Related Workmentioning

confidence: 99%

Error-Aware Imitation Learning from Teleoperation Data for Mobile Manipulation

Wong¹,

Tung²,

Kurenkov³

et al. 2021

Preprint

View full text Add to dashboard Cite

In mobile manipulation (MM), robots can both navigate within and interact with their environment and are thus able to complete many more tasks than robots only capable of navigation or manipulation. In this work, we explore how to apply imitation learning (IL) to learn continuous visuo-motor policies for MM tasks. Much prior work has shown that IL can train visuo-motor policies for either manipulation or navigation domains, but few works have applied IL to the MM domain. Doing this is challenging for two reasons: on the data side, current interfaces make collecting high-quality human demonstrations difficult, and on the learning side, policies trained on limited data can suffer from covariate shift when deployed. To address these problems, we first propose MOBILE MANIPULATION ROBOTURK (MOMART), a novel teleoperation framework allowing simultaneous navigation and manipulation of mobile manipulators, and collect a first-of-its-kind large scale dataset in a realistic simulated kitchen setting. We then propose a learned error detection system to address covariate shift by detecting when an agent is in a potential failure state. We train performant IL policies and error detectors from this data, and achieve over 45% task success rate and 85% error detection success rate across multiple multi-stage tasks when trained on expert data. Codebase, datasets, visualization, and more available at https://sites.google.com/view/il-for-mm/home.

show abstract

“…On the other hand, deep learning enables us to learn effective navigation policies from a large amount of experience. The recent development of high-performance navigation simulators, such as Habitat [5] and iGibson [6], has enabled researchers to develop large scale visual navigation algorithms [7], [8], [9], [10], [11], [12], [13], [14] for indoor environments. Wijmans et al [9] proposed an end-to-end vision-based reinforcement learning algorithm to train nearperfect agents that can navigate unseen indoor environments without access to the map by leveraging billions of simulation samples.…”

Section: A Visual Navigationmentioning

confidence: 99%

Learning to Navigate Sidewalks in Outdoor Environments

Sorokin

Tan

Liu

et al. 2021

Preprint

View full text Add to dashboard Cite

Outdoor navigation on sidewalks in urban environments is the key technology behind important human assistive applications, such as last-mile delivery or neighborhood patrol. This paper aims to develop a quadruped robot that follows a route plan generated by public map services, while remaining on sidewalks and avoiding collisions with obstacles and pedestrians. We devise a two-staged learning framework, which first trains a teacher agent in an abstract world with privileged ground-truth information, and then applies Behavior Cloning to teach the skills to a student agent who only has access to realistic sensors. The main research effort of this paper focuses on overcoming challenges when deploying the student policy on a quadruped robot in the real world. We propose methodologies for designing sensing modalities, network architectures, and training procedures to enable zeroshot policy transfer to unstructured and dynamic real outdoor environments. We evaluate our learning framework on a quadrupedal robot navigating sidewalks in the city of Atlanta, USA. Using the learned navigation policy and its onboard sensors, the robot is able to walk 3.2 kilometers with a limited number of human interventions.

show abstract

VOILA: Visual-Observation-Only Imitation Learning for Autonomous Navigation

Cited by 4 publications

References 13 publications

MIDGARD: A Simulation Platform for Autonomous Navigation in Unstructured Environments

MIDGARD: A Simulation Platform for Autonomous Navigation in Unstructured Environments

Error-Aware Imitation Learning from Teleoperation Data for Mobile Manipulation

Learning to Navigate Sidewalks in Outdoor Environments

Contact Info

Product

Resources

About