Bi-Directional Domain Adaptation for Sim2Real Transfer of Embodied Navigation Agents

Truong, Joanne; Chernova, Sonia; Batra, Dhruv

doi:10.1109/lra.2021.3062303

Cited by 51 publications

(19 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Education Because simulation provides the ability to train a large number of robots in parallel and provides rich data, Truong et al [353] used educational simulations before deploying the robots. They proposed bidirectional domain adaptation (BDA), an approach that connects the sim-vs-real gap in both directions for point goal navigation.…”

Section: ) Metaverse Applicationsmentioning

confidence: 99%

See 1 more Smart Citation

A Metaverse: Taxonomy, Components, Applications, and Open Challenges

2022

View full text Add to dashboard Cite

Unlike previous studies on the Metaverse based on Second Life, the current Metaverse is based on the social value of Generation Z that online and offline selves are not different. With the technological development of deep learning-based high-precision recognition models and natural generation models, Metaverse is being strengthened with various factors, from mobile-based always-on access to connectivity with reality using virtual currency. The integration of enhanced social activities and neural-net methods requires a new definition of Metaverse suitable for the present, different from the previous Metaverse. This paper divides the concepts and essential techniques necessary for realizing the Metaverse into three components (i.e., hardware, software, and contents) and three approaches (i.e., user interaction, implementation, and application) rather than marketing or hardware approach to conduct a comprehensive analysis. Furthermore, we describe essential methods based on three components and techniques to Metaverse's representative Ready Player One, Roblox, and Facebook research in the domain of films, games, and studies. Finally, we summarize the limitations and directions for implementing the immersive Metaverse as social influences, constraints, and open challenges.

show abstract

Section: ) Metaverse Applicationsmentioning

confidence: 99%

“…SocialOnaolapo et al[346], Bailey et al[347], Luria and Foulds[348], Do et al[349] Marketing Fernanda[350], Avadhanula et al[351], Sinha et al[352] Education Truong et al[353] …”

mentioning

confidence: 99%

A Metaverse: Taxonomy, Components, Applications, and Open Challenges

2022

View full text Add to dashboard Cite

show abstract

“…But simulated locomotion is hard to replicate on a real platform, even after hand-tuning, which emphasizes the importance of learning a transferable policy that is the focus of this work. Domain randomization [37], [38], [39] is a possible way to close the sim2real gap. Previous domain randomization efforts [37] often assume the system parameters are within a Gaussian distribution and use a non-differentiable physics engine.…”

Section: Related Workmentioning

confidence: 99%

A Recurrent Differentiable Engine for Modeling Tensegrity Robots Trainable with Low-Frequency Data

Wang¹,

Aanjaneya²,

Bekris³

2022

Preprint

View full text Add to dashboard Cite

“…Significant progress has been made in the problem of adapting policies trained with RL from a source to a target environment. Unlike RO-BUSTNAV, major assumptions involved in such transfer settings are either access to task-supervision in the target environment [24] or access to paired data from the source and target environments [23,54]. Domain Randomization (DR) [2,46,38,42] is another common approach to train policies robust to various environmental factors.…”

Section: Related Workmentioning

confidence: 99%

RobustNav: Towards Benchmarking Robustness in Embodied Navigation

Chattopadhyay

Hoffman

Mottaghi

et al. 2021

2021 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

As an attempt towards assessing the robustness of embodied navigation agents, we propose ROBUSTNAV, a framework to quantify the performance of embodied navigation agents when exposed to a wide variety of visual -affecting RGB inputs -and dynamics -affecting transition dynamics -corruptions. Most recent efforts in visual navigation have typically focused on generalizing to novel target environments with similar appearance and dynamics characteristics. With ROBUSTNAV, we find that some standard embodied navigation agents significantly underperform (or fail) in the presence of visual or dynamics corruptions. We systematically analyze the kind of idiosyncrasies that emerge in the behavior of such agents when operating under corruptions. Finally, for visual corruptions in ROBUSTNAV, we show that while standard techniques to improve robustness such as data-augmentation and self-supervised adaptation offer some zero-shot resistance and improvements in navigation performance, there is still a long way to go in terms of recovering lost performance relative to clean "non-corrupt" settings, warranting more research in this direction. Our code is available at https://github.com/allenai/robustnav.

show abstract

Bi-Directional Domain Adaptation for Sim2Real Transfer of Embodied Navigation Agents

Cited by 51 publications

References 15 publications

A Metaverse: Taxonomy, Components, Applications, and Open Challenges

A Metaverse: Taxonomy, Components, Applications, and Open Challenges

A Recurrent Differentiable Engine for Modeling Tensegrity Robots Trainable with Low-Frequency Data

RobustNav: Towards Benchmarking Robustness in Embodied Navigation

Contact Info

Product

Resources

About