iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks

Abstract

Recent research in embodied AI has been boosted by the use of simulation environments to develop and train robot learning approaches. However, the use of simulation has skewed the attention to tasks that only require what robotics simulators can simulate: motion and physical contact. We present iGibson 2.0, an open-source simulation environment that supports the simulation of a more diverse set of household tasks through three key innovations. First, iGibson 2.0 supports object states, including temperature, wetness level, cleanliness level, and toggled and sliced states, necessary to cover a wider range of tasks. Second, iGibson 2.0 implements a set of predicate logic functions that map the simulator states to logic states like Cooked or Soaked. Additionally, given a logic state, iGibson 2.0 can sample valid physical states that satisfy it. This functionality can generate potentially infinite instances of tasks with minimal effort from the users. The sampling mechanism allows our scenes to be more densely populated with small objects in semantically meaningful locations. Third, iGibson 2.0 includes a virtual reality (VR) interface to immerse humans in its scenes to collect demonstrations. As a result, we can collect demonstrations from humans on these new types of tasks, and use them for imitation learning. We evaluate the new capabilities of iGibson 2.0 to enable robot learning of novel tasks, in the hope of demonstrating the potential of this new simulator to support new directions of research in embodied AI. iGibson 2.0 and its new dataset are publicly available at http://svl.stanford.edu/igibson/.

Ditto: Building Digital Twins of Articulated Objects from Interaction

“…Physical simulators have become a vital tool for embodied AI research. A growing trend is shifting from static 3D scenes for visual navigation [23,46,59] to interactive environments that support physical interaction between the robot and the objects [9,24,52]. Interactive 3D assets are the key elements to construct these simulators.…”

“…Recent efforts in embodied AI platforms [23,24,52] have incorporated interactive articulated objects, such as cabinets and drawers, in simulated household environments and employed them for training virtual agents. Even so, they heavily rely on graphics designers and engineers to author and curate the object models, limiting the scalability of the asset acquisition process.…”

Learning to Act with Affordance-Aware Multimodal Neural SLAM

“…Recent progress in Embodied Artificial Intelligence, spans both simulation environments Kolve et al (2017); Li et al (2021); Savva et al (2019); Gan et al (2020); Puig et al (2018) and sophisticated tasks Das et al (2018); Anderson et al (2018); Shridhar et al (2020). Our work is most closely related to research in language-guided task completion, Neural SLAM, and exploration.…”

LUMINOUS: Indoor Scene Generation for Embodied AI Challenges

“…Embodied artificial intelligence (EAI) has attracted significant attention, both in advanced deep learning models and algorithms [1,2,3,4] and the rapid development of simulated platforms [5,6,7,8,9]. Many open challenges [10,11,12,13] have been proposed to facilitate EAI research.…”

“…Many open challenges [10,11,12,13] have been proposed to facilitate EAI research. A critical bottleneck in existing simulated platforms [10,12,8,5,14] is the limited number of indoor scenes that support vision-and-language navigation, object interaction, and complex household tasks. This limitation makes it difficult to verify whether state-of-the-art methods generalize well to unseen scenarios or whether they are specialized to a small number of room structures.…”