Feature discovery and visualization of robot mission data using convolutional autoencoders and Bayesian nonparametric topic models

Flaspohler, Genevieve; Roy, Nicholas; Girdhar, Yogesh

doi:10.1109/iros.2017.8202130

Cited by 9 publications

(17 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In vision based scientific exploration, semantic image representations have been explored for years as a useful and communication-efficient tool for mission summarization (e.g., [35]). One class of representations are "semantic segmentations", which are [92], showing a coral reef near Kaho'olawe.…”

Section: Semantic Image Representationsmentioning

confidence: 99%

“…Furthermore, some low-dimensional semantic representations are based on using features from pretrained deep neural networks, which represents another form of transfer learning [35,100].…”

Section: Transfer Learningmentioning

confidence: 99%

“…At the other extreme, "curious" robots use a generic unsupervised vision model and autonomously move towards anything in their environment that is surprising or novel to the model [43]; the lack of operator input makes it impossible to directly specify particular scientific objectives using this approach. [35,82]. However, these correlations are hard to model in the observation space (pixels), where even nearly identical images can be made distant by effects like sensor noise and slight changes in illumination [109].…”

Section: Path Planning For Autonomous Sciencementioning

confidence: 99%

“…ROST has been successfully used by a robotic explorer to enable online mission summarization [35] as well as unsupervised "curious" exploration [43].…”

Section: Path Planning For Autonomous Sciencementioning

confidence: 99%

See 3 more Smart Citations

Enabling human-robot cooperation in scientific exploration of bandwidth-limited environments

Jamieson¹

2020

View full text Add to dashboard Cite

None of this work would have been possible without the guidance and support of my co-supervisors, Dr. Yogesh Girdhar and Prof. Jonathan How. They have been exceedingly generous in giving their time, encouragement, and advice in matters both technical and related to professional development. I feel very grateful for the combined breadth and depth of their backgrounds, experiences, and technical knowledge, as this has been instrumental in allowing me to pursue such a cross-disciplinary and complex topic as co-robotic scientific exploration. I would next like to thank all of my collaborators and peers involved in the research and discussion of the work and ideas presented in this thesis. My fellow members of the WARPLab and the Autonomous Controls Lab always created a positive, supportive, and productive atmosphere to work in, one I missed when working from home due to current world affairs. I'd like to particularly thank Vv, John, Genevieve, Victoria, Mike, Kaveh, Kasra, and Kevin for many long discussions and useful advice on the topics and ideas presented in this work. I'd also like to thank Nathan, Brian, and Stefano for their invaluable technical advice and support. Alongside them are all the others at MIT, WHOI, and in the MIT-WHOI Joint Program who have been excellent friends to me since coming to MA and made me feel at home in a new country. Lastly, I would like to thank my family. In particular, my partner Victoria who provides me with constant support and inspires me with her incredible work ethic and dedication, as well my mom and dad and my brother Chris, whose love and encouragement have been ever-present and invaluable. I'd also like to thank Suzy and the rest of my extended family, who have likewise been major sources of support in both the best and worst of times. Finally, I dedicate this thesis in memory of my grandparents Chris & Molly, who were each deeply loved and are deeply missed; they always brought out the very best in me.

show abstract

Section: Semantic Image Representationsmentioning

confidence: 99%

“…Furthermore, some low-dimensional semantic representations are based on using features from pretrained deep neural networks, which represents another form of transfer learning [35,100].…”

Section: Transfer Learningmentioning

confidence: 99%

Section: Path Planning For Autonomous Sciencementioning

confidence: 99%

“…ROST has been successfully used by a robotic explorer to enable online mission summarization [35] as well as unsupervised "curious" exploration [43].…”

Section: Path Planning For Autonomous Sciencementioning

confidence: 99%

See 2 more Smart Citations

Enabling human-robot cooperation in scientific exploration of bandwidth-limited environments

Jamieson¹

2020

View full text Add to dashboard Cite

show abstract

“…Finally, the combined hybrid approach is evaluated on imagery from two different marine missions collected by the SeaBED AUV at the Hannibal Sea Mount off the coast of Panama (Section 4.4), followed by a discussion of the results (Section 4.5). The work presented in this chapter has appeared in abbreviated form in Flaspohler et al[34].…”

mentioning

confidence: 99%

Untitled

Flaspohler¹

Statistical Models and Decision Making for Robotic Scientific Information Gathering

View full text Add to dashboard Cite

Mobile robots and autonomous sensors have seen increasing use in scientific applications, from planetary rovers surveying for signs of life on Mars, to environmental buoys measuring and logging oceanographic conditions in coastal regions. This thesis makes contributions in both planning algorithms and model design for autonomous scientific information gathering, demonstrating how theory from machine learning, decision theory, theory of optimal experimental design, and statistical inference can be used to develop online algorithms for robotic information gathering that are robust to modeling errors, account for spatiotemporal structure in scientific data, and have probabilistic performance guarantees.This thesis first introduces a novel sample selection algorithm for online, irrevocable sampling in data streams that have spatiotemporal structure, such as those that commonly arise in robotics and environmental monitoring. Given a limited sampling capacity, the proposed periodic secretary algorithm uses an information-theoretic reward function to select samples in real-time that maximally reduce posterior uncertainty in a given scientific model. Additionally, we provide a lower bound on the quality of samples selected by the periodic secretary algorithm by leveraging the submodularity of the information-theoretic reward function. Finally, we demonstrate the robustness of the proposed approach by employing the periodic secretary algorithm to select samples irrevocably from a seven-year oceanographic data stream collected at the Martha's Vineyard Coastal Observatory off the coast of Cape Cod, USA.Secondly, we consider how scientific models can be specified in environmentssuch as the deep sea or deep space -where domain scientists may not have enough a priori knowledge to formulate a formal scientific model and hypothesis. These domains require scientific models that start with very little prior information and construct a model of the environment online as observations are gathered. We propose unsupervised machine learning as a technique for science model-learning in these environments. To this end, we introduce a hybrid Bayesian-deep learning model that learns a nonparametric topic model of a visual environment. We use this semantic visual model to identify observations that are poorly explained in the current model,

show abstract

Learning features from georeferenced seafloor imagery with location guided autoencoders

Yamada

Prügel-Bennett

Thornton

2020

Journal of Field Robotics

View full text Add to dashboard Cite

Although modern machine learning has the potential to greatly speed up the interpretation of imagery, the varied nature of the seabed and limited availability of expert annotations form barriers to its widespread use in seafloor mapping applications. This motivates research into unsupervised methods that function without large databases of human annotations. This paper develops an unsupervised feature learning method for georeferenced seafloor visual imagery that considers patterns both within the footprint of a single image frame and broader scale spatial characteristics. Features within images are learnt using an autoencoder developed based on the AlexNet deep convolutional neural network. Features larger than each image frame are learnt using a novel loss function that regularises autoencoder training using the Kullback–Leibler divergence function to loosely assume that images captured within a close distance of each other look more similar than those that are far away. The method is used to semantically interpret images taken by an autonomous underwater vehicle at the Southern Hydrates Ridge, an active gas hydrate field and site of a seafloor cabled observatory at a depth of 780 m. The method's performance when applied to clustering and content‐based image retrieval is assessed against a ground truth consisting of more than 18,000 human annotations. The study shows that the location based loss function increases the rate of information retrieval by a factor of two for seafloor mapping applications. The effects of physics‐based colour correction and image rescaling are also investigated, showing that the improved consistency of spatial information achieved by rescaling is beneficial for recognising artificial objects such as cables and infrastructures, but is less effective for natural objects that have greater dimensional variability.

show abstract

Feature discovery and visualization of robot mission data using convolutional autoencoders and Bayesian nonparametric topic models

Cited by 9 publications

References 18 publications

Enabling human-robot cooperation in scientific exploration of bandwidth-limited environments

Enabling human-robot cooperation in scientific exploration of bandwidth-limited environments

Untitled

Learning features from georeferenced seafloor imagery with location guided autoencoders

Contact Info

Product

Resources

About