Mono Lay out: Amodal scene layout from a single image

AutoLay: Benchmarking amodal layout estimation for autonomous driving

“…In earlier work, we presented MonoLayout [3]-perhaps the first approach to amodally reason about the static and dynamic layout of an urban driving scene. While the interest in this nascent area is rapidly increasing, the absence of standardized task specification and evaluation have been slowing down research.…”

“…While most datasets have lidar scans that often span a larger sensor swath, such lidar beams are not dense enough to perceive thin scene structures. Approaches such as MonoLayout [3] compensate for this lack of precision training data by adversarial learning, using OpenStreetMap [22] data. More recently, Wang et al [10] release a dataset that provides a parametric bird's eye view representation for images from the KITTI [1] and nuScenes [20] datasets, but the topview representations are synthesized views of a simulated parametric surface model.…”

“…Amodal scene layout estimation is an emerging area of study where prior efforts [3], [4], [5] have only focused on amodal completion for coarse categories such as roads and sidewalks. While recent efforts have incorporated dynamic objects [3], they fall short of modeling higher-level behaviors in urban driving scenarios such as lane-keeping, merging, etc. Further, each of these efforts [3], [4], [5] lack consensus in problem specifications and evaluation protocols.…”

“…While recent efforts have incorporated dynamic objects [3], they fall short of modeling higher-level behaviors in urban driving scenarios such as lane-keeping, merging, etc. Further, each of these efforts [3], [4], [5] lack consensus in problem specifications and evaluation protocols.…”

“…AutoLay includes 44 sequences from KITTI [1] and 89 sequences from the Argoverse [2] datasets. We implement several baselines and prior art, and open-source our code, models, and data [3], [4], [6]. We additionally propose VideoLayout, a simple approach that achieves top performance on the AutoLay benchmark by leveraging temporal information across image sequences, advancing the state-of-the-art of amodal layout estimation.…”