Approximate Distributed Spatiotemporal Topic Models for Multi-Robot Terrain Characterization

Doherty, Kevin; Flaspohler, Genevieve; Roy, Nicholas; Girdhar, Yogesh

doi:10.1109/iros.2018.8594442

Cited by 11 publications

(19 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While there has been progress in other unsupervised semantic image segmentation approaches [21], including deeplearning based methods [22], by not leveraging these correlations they produce lower quality maps of large environments. Doherty et al [23] showed that repeatedly matching the topic models of two robots with the Hungarian algorithm [24], merging them together, and distributing the merged model would result in both robots converging to a single set of good semantic labels, even if this was done at a low frequency. To our knowledge, [23] is the only prior work that has explored multi-robot distributed semantic mapping with representations learned online.…”

Section: Related Workmentioning

confidence: 99%

Multi-Robot Distributed Semantic Mapping in Unfamiliar Environments through Online Matching of Learned Representations

Jamieson¹,

Fathian²,

Khosoussi³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We present a solution to multi-robot distributed semantic mapping of novel and unfamiliar environments. Most state-of-the-art semantic mapping systems are based on supervised learning algorithms that cannot classify novel observations online. While unsupervised learning algorithms can invent labels for novel observations, approaches to detect when multiple robots have independently developed their own labels for the same new class are prone to erroneous or inconsistent matches. These issues worsen as the number of robots in the system increases and prevent fusing the local maps produced by each robot into a consistent global map, which is crucial for cooperative planning and joint mission summarization. Our proposed solution overcomes these obstacles by having each robot learn an unsupervised semantic scene model online and use a multiway matching algorithm to identify consistent sets of matches between learned semantic labels belonging to different robots. Compared to the state of the art, the proposed solution produces 20-60% higher quality global maps that do not degrade even as many more local maps are fused. OPEN SOURCE SOFTWAREAn implementation of this solution is contributed in the "Sunshine" 3D semantic mapping ROS package, a general purpose single-and multi-robot semantic mapping system: https://gitlab.com/warplab/ros/sunshine.

show abstract

Section: Related Workmentioning

confidence: 99%

Multi-Robot Distributed Semantic Mapping in Unfamiliar Environments through Online Matching of Learned Representations

Jamieson¹,

Fathian²,

Khosoussi³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Thus, the robots are likely to use a variety of distinct semantic labels to represent the same phenomenon, especially if there are many different phonomena in the environment. Some previous work which started to address this issue explored sharing the topic-word distribution matrices between robots regularly, thus ensuring that all robots would eventually converge towards shared semantic representations [23].…”

Section: Multi-robot Federated Explorationmentioning

confidence: 99%

“…This enables the robots to communicate labels and observations among each other, and potentially still merge topic models if appropriate, with fewer drawbacks and less communication required. Of particular interest is using the CLEAR algorithm [33], which is capable of efficiently solving the multi-agent data association problem to find all sets of equivalent semantic labels across all robots, to find much better correspondences than the 1-to-1 correspondences solved using the Hungarian algorithm in [23].…”

Section: Multi-robot Federated Explorationmentioning

confidence: 99%

Enabling human-robot cooperation in scientific exploration of bandwidth-limited environments

Jamieson¹

2020

View full text Add to dashboard Cite

None of this work would have been possible without the guidance and support of my co-supervisors, Dr. Yogesh Girdhar and Prof. Jonathan How. They have been exceedingly generous in giving their time, encouragement, and advice in matters both technical and related to professional development. I feel very grateful for the combined breadth and depth of their backgrounds, experiences, and technical knowledge, as this has been instrumental in allowing me to pursue such a cross-disciplinary and complex topic as co-robotic scientific exploration. I would next like to thank all of my collaborators and peers involved in the research and discussion of the work and ideas presented in this thesis. My fellow members of the WARPLab and the Autonomous Controls Lab always created a positive, supportive, and productive atmosphere to work in, one I missed when working from home due to current world affairs. I'd like to particularly thank Vv, John, Genevieve, Victoria, Mike, Kaveh, Kasra, and Kevin for many long discussions and useful advice on the topics and ideas presented in this work. I'd also like to thank Nathan, Brian, and Stefano for their invaluable technical advice and support. Alongside them are all the others at MIT, WHOI, and in the MIT-WHOI Joint Program who have been excellent friends to me since coming to MA and made me feel at home in a new country. Lastly, I would like to thank my family. In particular, my partner Victoria who provides me with constant support and inspires me with her incredible work ethic and dedication, as well my mom and dad and my brother Chris, whose love and encouragement have been ever-present and invaluable. I'd also like to thank Suzy and the rest of my extended family, who have likewise been major sources of support in both the best and worst of times. Finally, I dedicate this thesis in memory of my grandparents Chris & Molly, who were each deeply loved and are deeply missed; they always brought out the very best in me.

show abstract

“…For example, the recovery of soundtraps for biological research in Figure 1-3, and the dangerous task of inspecting ship hulls for mines (which has previously been performed by either trained human divers or marine mammals). Additionally, there has been increasing interest in the use of robots for semantic-level biological surveying, as in prior work which examines online learning for multiple communication-constrained underwater vehicles [12].…”

Section: Motivationmentioning

confidence: 99%

Robust non-Gaussian semantic simultaneous localization and mapping

Doherty¹

2019

View full text Add to dashboard Cite

The recent success of object detection systems motivates object-based representations for robot navigation; i.e. semantic simultaneous localization and mapping (SLAM), in which we aim to jointly estimate the pose of the robot over time as well as the location and semantic class of observed objects. A solution to the semantic SLAM problem necessarily addresses the continuous inference problems where am I? and where are the objects?, but also the discrete inference problem what are the objects?. We consider the problem of semantic SLAM under non-Gaussian uncertainty. The most prominent case in which this arises is from data association uncertainty, where we do not know with certainty what objects in the environment caused the measurement made by our sensor. The semantic class of an object can help to inform data association; a detection classified as a door is unlikely to be associated to a chair object. However, detectors are imperfect, and incorrect classification of objects can be detrimental to data association. While previous approaches seek to eliminate such measurements, we instead model the robot and landmark state uncertainty induced by data association in the hopes that new measurements may disambiguate state estimates, and that we may provide representations useful for developing decisionmaking strategies where a robot can take actions to mitigate multimodal uncertainty. The key insight we leverage is that the semantic SLAM problem with unknown data association can be reframed as a non-Gaussian inference problem. We present two solutions to the resulting problem: we first assume Gaussian measurement models, and non-Gaussianity only due to data association uncertainty. We then relax this assumption and provide a method that can cope with arbitrary non-Gaussian measurement models. We show quantitatively on both simulated and real data that both proposed methods have robustness advantages as compared to traditional solutions when data associations are uncertain. I would first and foremost like to thank Professor John Leonard for giving me the opportunity to join the Marine Robotics Group at MIT and being a constant source of inspiration and positivity in robotics. His advice, encouragement, and support made the work in this thesis possible. It is because of him that I first learned of the MIT-WHOI Joint Program, for which I am incredibly grateful. I would also like to thank the members of the Marine Robotics Group and Robust Robotics Group, especially Prof. Nicholas Roy for giving me a "home base" at MIT in the Robust Robotics Group during my first year. It's hard to imagine a better group of folks to chat with about research, and I appreciate so much the stimulating research discussion, positivity, and support within these groups. I feel so lucky to have had the opportunity to learn from them. I'd like to thank my MIT-WHOI Joint Program cohort for being great friends and really making Cambridge and Woods Hole my home for the past two years. I'd also like to acknowledge the program more broadly for giving m...

show abstract

Approximate Distributed Spatiotemporal Topic Models for Multi-Robot Terrain Characterization

Cited by 11 publications

References 12 publications

Multi-Robot Distributed Semantic Mapping in Unfamiliar Environments through Online Matching of Learned Representations

Multi-Robot Distributed Semantic Mapping in Unfamiliar Environments through Online Matching of Learned Representations

Enabling human-robot cooperation in scientific exploration of bandwidth-limited environments

Robust non-Gaussian semantic simultaneous localization and mapping

Contact Info

Product

Resources

About