MotionSC: Data Set and Network for Real-Time Semantic Mapping in Dynamic Environments

Wilson, Joseph; Song, Jingyu; Yuewei, Fu,; Zhang, Arthur; Andrew, Capodieci,; Jayakumar, Paramsothy; Barton, Kira; Ghaffari, Maani

doi:10.1109/lra.2022.3188435

Cited by 19 publications

(4 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Semantics are important to robot perception for better scene understanding and interaction [6]. Whereas many semantic mapping works have explored learning-based local mapping [12]- [17], our work is a mathematically-derived 3D global mapping algorithm. Additionally, our method builds upon existing deep learning research by directly taking the output from neural networks as input, instead of attempting to embed all information within a latent space.…”

Section: A Semantic Mappingmentioning

confidence: 99%

“…We generate a synthetic multi-view scene completion data set sequence from the CARLA [66] simulator. The methodology for its creation is available publicly in [17]. We generate ground-truth environment models by uniformly distributing multiple LiDAR sensors around the ego vehicle, effectively obtaining a 3D Monte Carlo sampling of the world which is i.i.d.…”

Section: Carla Data Setmentioning

confidence: 99%

See 1 more Smart Citation

Dynamic Semantic Occupancy Mapping Using 3D Scene Flow and Closed-Form Bayesian Inference

et al. 2022

Self Cite

View full text Add to dashboard Cite

This paper reports on a dynamic semantic mapping framework that incorporates 3D scene flow measurements into a closed-form Bayesian inference model. Existence of dynamic objects in the environment can cause artifacts and traces in current mapping algorithms, leading to an inconsistent map posterior. We leverage state-of-the-art semantic segmentation and 3D flow estimation using deep learning to provide measurements for map inference. We develop a Bayesian model that propagates the scene with flow and infers a 3D continuous (i.e., can be queried at arbitrary resolution) semantic occupancy map outperforming its static counterpart. Extensive experiments using publicly available data sets show that the proposed framework improves over its predecessors and input measurements from deep neural networks consistently.

show abstract

Section: A Semantic Mappingmentioning

confidence: 99%

Section: Carla Data Setmentioning

confidence: 99%

Dynamic Semantic Occupancy Mapping Using 3D Scene Flow and Closed-Form Bayesian Inference

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…In particular, a reliable 3D dynamic object tracking can increase the success rate on hardware. Another possible direction is to develop a real-time world model that takes into account the dynamics and semantics of the scene [44], [45].…”

Section: Limitations and Future Workmentioning

confidence: 99%

SoLo T-DIRL: Socially-Aware Dynamic Local Planner based on Trajectory-Ranked Deep Inverse Reinforcement Learning

Xu¹,

Chakhachiro²,

Kathuria³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

This work proposes a new framework for a socially-aware dynamic local planner in crowded environments by building on the recently proposed Trajectory-ranked Maximum Entropy Deep Inverse Reinforcement Learning (T-MEDIRL). To address the social navigation problem, our multi-modal learning planner explicitly considers social interaction factors, as well as social-awareness factors into T-MEDIRL pipeline to learn a reward function from human demonstrations. Moreover, we propose a novel trajectory ranking score using the sudden velocity change of pedestrians around the robot to address the sub-optimality in human demonstrations.Our evaluation shows that this method can successfully make a robot navigate in a crowded social environment and outperforms the state-of-art social navigation methods in terms of the success rate, navigation time, and invasion rate.

show abstract

“…Simultaneous Localization and Mapping (SLAM) is a fundamental mobile robotics problem in which a robot constructs a map of its environment and localizes itself within the map. Previous SLAM approaches have focused on using particle filters [1,2], extended Kalman filters [3], and graph-based optimization methods [4,5] to complete localization, but recently the focus has shifted to applying machine learning to graph-based SLAM techniques [6,7].…”

Section: Introductionmentioning

confidence: 99%

Volume-DROID: A Real-Time Implementation of Volumetric Mapping with DROID-SLAM

Stratton,

Sai Garimella,

Saxena

et al. 2023

AAIML

View full text Add to dashboard Cite

This paper presents Volume-DROID, a novel approach for Simultaneous Localization and Mapping (SLAM) that integrates Volumetric Mapping and Differentiable Recurrent Optimization-Inspired Design (DROID). Volume-DROID takes camera images (monocular or stereo) or frames from a video as input and combines DROID-SLAM, point cloud registration, an off-the-shelf semantic segmentation network, and Convolutional Bayesian Kernel Inference (ConvBKI) to generate a 3D semantic map of the environment and provide accurate localization for the robot. The key innovation of our method is the real-time fusion of DROID-SLAM and Convolutional Bayesian Kernel Inference (ConvBKI), achieved through the introduction of point cloud generation from RGB-Depth frames and optimized camera poses. This integration, engineered to enable efficient and timely processing, minimizes lag and ensures effective performance of the system. Our approach facilitates functional real-time online semantic mapping with just camera images or stereo video input. Our paper offers an open-source Python implementation of the algorithm, available at https://github.com/ peterstratton/Volume-DROID.

show abstract

MotionSC: Data Set and Network for Real-Time Semantic Mapping in Dynamic Environments

Cited by 19 publications

References 40 publications

Dynamic Semantic Occupancy Mapping Using 3D Scene Flow and Closed-Form Bayesian Inference

Dynamic Semantic Occupancy Mapping Using 3D Scene Flow and Closed-Form Bayesian Inference

SoLo T-DIRL: Socially-Aware Dynamic Local Planner based on Trajectory-Ranked Deep Inverse Reinforcement Learning

Volume-DROID: A Real-Time Implementation of Volumetric Mapping with DROID-SLAM

Contact Info

Product

Resources

About