We present Habitat, a platform for research in embodied artificial intelligence (AI). Habitat enables training embodied agents (virtual robots) in highly efficient photorealistic 3D simulation. Specifically, Habitat consists of: (i) Habitat-Sim: a flexible, high-performance 3D simulator with configurable agents, sensors, and generic 3D dataset handling. Habitat-Sim is fast -when rendering a scene from Matterport3D, it achieves several thousand frames per second (fps) running single-threaded, and can reach over 10,000 fps multi-process on a single GPU. (ii) Habitat-API: a modular high-level library for end-toend development of embodied AI algorithms -defining tasks (e.g. navigation, instruction following, question answering), configuring, training, and benchmarking embodied agents.These large-scale engineering contributions enable us to answer scientific questions requiring experiments that were till now impracticable or 'merely' impractical. Specifically, in the context of point-goal navigation: (1) we revisit the comparison between learning and SLAM approaches from two recent works [20,16] and find evidence for the opposite conclusion -that learning outperforms SLAM if scaled to an order of magnitude more experience than previous investigations, and (2) we conduct the first cross-dataset generalization experiments {train, test} × {Matterport3D, Gibson} for multiple sensors {blind, RGB, RGBD, D} and find that only agents with depth (D) sensors generalize across datasets. We hope that our open-source platform and these findings will advance research in embodied AI. * Denotes equal contribution.

Figure 1: Simulation of a peach tree with anatomically realistic geometry (Prunus Persica), with fracture. Peaches fall from the tree swaying in the space-time Perlin wind. 299,707 triangles, 237 branches, 3,556 twigs, 18,536 leaves, 330 fruits, 2,950 reduced DOFs, 7 hierarchy levels, 5 msec of simulation per graphical frame. AbstractPhysically based simulation can produce quality motion of plants, but requires an authoring stage to convert plant "polygon soup" triangle meshes to a format suitable for physically based simulation. We give a system that can author complex simulation-ready plants in a manner of minutes. Our system decomposes the plant geometry, establishes a hierarchy, builds and connects simulation meshes, and detects instances. It scales to anatomically realistic geometry of adult plants, is robust to non-manifold input geometry, gaps between branches or leaves, free-flying leaves not connected to any branch, spurious geometry, and plant self-collisions in the input configuration. We demonstrate the results using a FEM model reduction simulator that can compute large-deformation dynamics of complex plants at interactive rates, subject to user forces, gravity or randomized wind. We also provide plant fracture (with prespecified patterns), inverse kinematics to easily pose plants, as well as interactive design of plant material properties. We authored and simulated over 100 plants from diverse climates and geographic regions, including broadleaf (deciduous) trees and conifers, bushes and flowers. Our largest simulations involve anatomically realistic adult trees with hundreds of branches and over 100,000 leaves.

This paper shows a method to extend 3D nonlinear elasticity model reduction to open-loop multi-level reduced deformable structures. Given a volumetric mesh, we decompose the mesh into several subdomains, build a reduced deformable model for each domain, and connect the domains using inertia coupling. This makes model reduction deformable simulations much more versatile: localized deformations can be supported without prohibitive computational costs, parts can be re-used and precomputation times shortened. Our method does not use constraints, and can handle large domain rigid body motion in addition to large deformations, due to our derivation of the gradient and Hessian of the rotation matrix in polar decomposition. We show real-time examples with multi-level domain hierarchies and hundreds of reduced degrees of freedom.