Mars Image Content Classification: Three Years of NASA Deployment and Recent Advances

Wagstaff, Kiri L.; Lu, Shuaiyao; Dunkel, Emily; Grimes, Kevin; Zhao, Brandon; Cai, Jesse; Cole, S.; Doran, Gary; Francis, Raymond W; Lee, Jake; Mandrake, Lukas

doi:10.48550/arxiv.2102.05011

Cited by 3 publications

(7 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These datasets have been used to train classifiers to detect classes of interest [18,24]. Development of annotated datasets allowed for deployment of multilabel Convolutional Neural Networks (CNNs) for classification of both science and engineering tasks for individual missions [15,23] which showed performance improvements over Support Vector Machine (SVM) classifiers. Additional planetary applications include the Science Captioning of Terrain Images (SCOTI) [19] which generates text captions for terrain images.…”

Section: Planetary Computer Visionmentioning

confidence: 99%

“…Terrain segmentation has been necessary for a rovers ability to navigate and maneuver autonomously, and it involves identifying features across the surface's terrain e.g., soil, sand, rocks, etc. Automated pipelines that utilize deep learning (DL) can enable the efficient (pixel-wise) classification of large volumes of extra-terrestrial images [23] for use in downstream tasks and analyses. DL is notoriously sample inefficient [13], often requiring on the order of 10,000 examples to achieve good performance or retraining when moving across different domains (e.g., different missions, planets, seasons lighting conditions, etc.)…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Mixed-domain Training Improves Multi-Mission Terrain Segmentation

Vincent¹,

Yepremyan²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

Planetary rover missions must utilize machine learning-based perception to continue extra-terrestrial exploration with little to no human presence. Martian terrain segmentation has been critical for rover navigation and hazard avoidance to perform further exploratory tasks, e.g. soil sample collection and searching for organic compounds. Current Martian terrain segmentation models require a large amount of labeled data to achieve acceptable performance, and also require retraining for deployment across different domains, i.e. different rover missions, or different tasks, i.e. geological identification and navigation. This research proposes a semi-supervised learning approach that leverages unsupervised contrastive pretraining of a backbone for a multi-mission semantic segmentation for Martian surfaces. This model will expand upon the current Martian segmentation capabilities by being able to deploy across different Martian rover missions for terrain navigation, by utilizing a mixeddomain training set that ensures feature diversity. Evaluation results of using average pixel accuracy show that a semi-supervised mixed-domain approach improves accuracy compared to single domain training and supervised training by reaching an accuracy of 97% for the Mars Science Laboratory's Curiosity Rover and 79.6% for the Mars 2020 Perseverance Rover. Further, providing different weighting methods to loss functions improved the models correct predictions for minority or rare classes by over 30% using the recall metric compared to standard cross-entropy loss. These results can inform future multi-mission and multi-task semantic segmentation for rover missions in a data-efficient manner.

show abstract

Section: Planetary Computer Visionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Mixed-domain Training Improves Multi-Mission Terrain Segmentation

Vincent¹,

Yepremyan²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Supervised learning for Mars images. Wagstaff et al proposed a fully supervised approach to training AlexNet-based [14] Mars classification models [25,24]. In these works, benchmark datasets were created to validate the classification performance of the trained models: the Mars Science Laboratory (MSL) dataset, and the High Resolution Imaging Science Experiment (HiRISE) dataset.…”

Section: Related Workmentioning

confidence: 99%

“…This dataset is used for contrastive learning in the same domain as the Perseverance rover images. The labeled dataset (herein referred to as MSL v2.1) was compiled and annotated by Wagstaff et al's [25,24] and consists of 6820 images divided into 19 classes of interest. The MSL v2.1 dataset is further divided by Sol into train, validation, and test subsets (see Table 1).…”

Section: Curiosity Rover Imagesmentioning

confidence: 99%

“…Existing DL-based approaches have tackled the aforementioned challenge using supervised (transfer) learning and typically require thousands of annotated images to achieve reasonable performance [25,24]. Although a manual labeling approach may seem viable, Mars images contain subtle class differences and fine-grained geological features that require highly specialized scientific knowledge and expertise, meaning that scaling manual efforts to the volume of required data is difficult, if not impractical.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Improving Contrastive Learning on Visually Homogeneous Mars Rover Images

Ward¹,

Moore²,

Pak³

et al. 2022

Preprint

View full text Add to dashboard Cite

Contrastive learning has recently demonstrated superior performance to supervised learning, despite requiring no training labels. We explore how contrastive learning can be applied to hundreds of thousands of unlabeled Mars terrain images, collected from the Mars rovers Curiosity and Perseverance, and from the Mars Reconnaissance Orbiter. Such methods are appealing since the vast majority of Mars images are unlabeled as manual annotation is labor intensive and requires extensive domain knowledge. Contrastive learning, however, assumes that any given pair of distinct images contain distinct semantic content. This is an issue for Mars image datasets, as any two pairs of Mars images are far more likely to be semantically similar due to the lack of visual diversity on the planet's surface. Making the assumption that pairs of images will be in visual contrast -when they are in fact not -results in pairs that are falsely considered as negatives, impacting training performance. In this study, we propose two approaches to resolve this: 1) an unsupervised deep clustering step on the Mars datasets, which identifies clusters of images containing similar semantic content and corrects false negative errors during training, and 2) a simple approach which mixes data from different domains to increase visual diversity of the total training dataset. Both cases reduce the rate of false negative pairs, thus minimizing the rate in which the model is incorrectly penalized during contrastive training. These modified approaches remain fully unsupervised end-to-end. To evaluate their performance, we add a single linear layer trained to generate class predictions based on these contrastively-learned features and demonstrate increased performance compared to supervised models; observing an improvement in classification accuracy of 3.06% using only 10% of the labeled data.

show abstract

Planet Four: A Neural Network's Search For Polar Spring-time Fans On Mars

McDonnell¹,

Jones²,

Schwamb³

et al. 2022

Preprint

View full text Add to dashboard Cite

Dark deposits visible from orbit appear in the Martian south polar region during the springtime. These are thought to form from explosive jets of carbon dioxide gas breaking through the thawing seasonal ice cap, carrying dust and dirt which is then deposited onto the ice as dark 'blotches', or blown by the surface winds into streaks or 'fans'. We investigate machine learning (ML) methods for automatically identifying these seasonal features in High Resolution Imaging Science Experiment (HiRISE) satellite imagery. We designed deep Convolutional Neural Networks (CNNs) that were trained and tested using the catalog generated by Planet Four, an online citizen science project mapping the south polar seasonal deposits. We validated the CNNs by comparing their results with those of ISODATA (Iterative Self-Organizing Data Analysis Technique) clustering and as expected, the CNNs were significantly better at predicting the results found by Planet Four, in both the area of predicted seasonal deposits and in delineating their boundaries. We found neither the CNNs or ISODATA were suited to predicting the source point and directions of seasonal fans, which is a strength of the citizen science approach. The CNNs showed good agreement with Planet Four in cross-validation metrics and detected some seasonal deposits in the HiRISE images missed in the Planet Four catalog; the total area of seasonal deposits predicted by the CNNs was 27% larger than that of the Planet Four catalog, but this aspect varied considerably on a per-image basis.

show abstract

Mars Image Content Classification: Three Years of NASA Deployment and Recent Advances

Cited by 3 publications

References 11 publications

Mixed-domain Training Improves Multi-Mission Terrain Segmentation

Mixed-domain Training Improves Multi-Mission Terrain Segmentation

Improving Contrastive Learning on Visually Homogeneous Mars Rover Images

Planet Four: A Neural Network's Search For Polar Spring-time Fans On Mars

Contact Info

Product

Resources

About