Mars Image Content Classification: Three Years of NASA Deployment and Recent Advances

Wagstaff, Kiri L.; Lu, Steven; Dunkel, Emily; Grimes, Kevin; Zhao, Brandon; Cai, Jesse; Cole, S.; Doran, Gary; Francis, Raymond; Lee, Jake; Mandrake, Lukas

doi:10.1609/aaai.v35i17.17784

Cited by 18 publications

(6 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A more rigorous parameter search will be undertaken in preparation for deployment. We employ a sol-based split as proposed by Wagstaff et al (2021) to evaluate generalization in a realistic setting, wherein training occurs on past data and validation/testing on future data. The term "sol" here refers to a measure of one Mars day.…”

Section: Methodsmentioning

confidence: 99%

“…Note that the solbased split reveals the temporal label shift between train and validation/test set, reflected in the gap between their accuracy across both deep CNNs reported in Table 1. Wagstaff et al (2021) provide a full description of the dataset generation process with detailed class distributions. A batch size of 80 was used in combination with an Adam optimizer (Kingma and Ba 2015) to train all the classifiers.…”

Section: Methodsmentioning

confidence: 99%

“…We describe our plan to deploy the system by replacing the non-interpretable counterpart currently hosted on the Atlas with the interpretable content-based classifier proposed in this paper. The MSL surface dataset 1 used in this paper was published by Wagstaff et al (2021).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Interactive Mars Image Content-Based Search with Interpretable Machine Learning

Vasu,

Lu,

Dunkel

et al. 2024

AAAI

View full text Add to dashboard Cite

The NASA Planetary Data System (PDS) hosts millions of images of planets, moons, and other bodies collected throughout many missions. The ever-expanding nature of data and user engagement demands an interpretable content classification system to support scientific discovery and individual curiosity. In this paper, we leverage a prototype-based architecture to enable users to understand and validate the evidence used by a classifier trained on images from the Mars Science Laboratory (MSL) Curiosity rover mission. In addition to providing explanations, we investigate the diversity and correctness of evidence used by the content-based classifier. The work presented in this paper will be deployed on the PDS Image Atlas, replacing its non-interpretable counterpart.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Interactive Mars Image Content-Based Search with Interpretable Machine Learning

Vasu,

Lu,

Dunkel

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…Data we get from LRO's Narrow Angle Cameras are 5-km swaths, at nominal orbit, so we perform a saliency detection step to nd surface features of interest. A detector developed for Mars HiRISE [9] worked well for our purposes, a er updating based on LROC image resolution. We use this detector to create a set of image chipouts (small cutouts) from the larger image, sampling the lunar globe.…”

Section: Lro Datamentioning

confidence: 99%

LROCNet: Detecting Impact Ejecta and Older Craters on the Lunar Surface

Dunkel

Grimes

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

NASA's Planetary Data System (PDS)* contains data collected by missions to explore our solar system. This includes Lunar Reconnaissance Orbiter (LRO), which has collected as much data as all other planetary missions combined. Currently, PDS offers no way to search lunar images based on content. Working with the PDS Cartography and Imaging Sciences Node (IMG), we develop LROCNet, a deep learning (DL) classifier for imagery from LRO's Narrow Angle Cameras (NACs). Data we get from NACs are 5km swaths, at nominal orbit, so we perform a saliency detection step to find surface features of interest. A detector developed for Mars HiRISE (Wagstaff et al, 2021) worked well for our purposes, after updating based on LROC image resolution. We use this detector to create a set of image chipouts (small cutouts) from the larger image, sampling the lunar globe. The chipouts are used to train LROCNet. We select classes of interest based on what is visible at the NAC resolution, consulting with scientists and performing a literature review. Initially, we had 7 classes: fresh crater, old crater, overlapping craters, irregular mare patches, rockfalls and landfalls, of scientific interest, and none. Using the Zooniverse platform, we set up a labeling tool and labeled 5,000 images. We found that fresh crater made up 11% of the data, old crater 18%, with the vast majority none. Due to limited examples of the other classes, we reduced our initial class set to: fresh crater (with impact ejecta), old crater, and none. We divided the images into train/validation/test set making sure no image swaths span multiple sets and fine tuned pre-trained DL models. VGG-11, a standard DL model, gives the best performance on the validation set, with an overall accuracy of 82% on the test set. We had 83% label agreement in our human label study; labeling was difficult as there is no clear class boundary. Our DL model accuracy is similar to human labelers.

show abstract

“…To help users to automatically discover Mars images of interest from the Planetary Data System (PDS) image atlas, the AlexNet trained on Earth images was finetuned and used for classifying twenty-four classes of Mars rover images and six classes (i.e., craters, bright sand dunes, dark sand dunes, dark slope streaks, other and edge) of HiRISE orbital images [29]. Based on the same method developed in [29], three new classes of interest, which are impact ejecta, spider, and swiss cheese were added in [30]. A novel software with the capability of Soil Property and Object Classification (SPOC) was proposed for classifying HiRISE orbital images and the Curiosity's Navigation Camera images.…”

Section: Introductionmentioning

confidence: 99%

MarsMapNet: A Novel Superpixel-Guided Multiview Feature Fusion Network for Efficient Martian Landform Mapping

Zhao,

Liu,

Tong

et al. 2024

IEEE Trans. Geosci. Remote Sensing

View full text Add to dashboard Cite

Abstract-Landform classification and mapping of the Martian surface using Mars orbiter images can provide an important reference for landing site selection and rovers' traversability evaluation in Mars exploration. Moreover, specific Martian landforms are closely associated with the evidences of water-related activities and Martian life, thus have crucial research importance. This article proposes a novel superpixelguided multi-view feature fusion network (MarsMapNet) for efficient mapping of the Martian landforms. In particular, the proposed MarsMapNet first generates the superpixel-level segments from Mars orbiter images by considering local morphological homogeneity of landforms. Then a multi-view feature extraction and fusion (MVF) network is developed, where abstract convolutional features are extracted based on scene-level patches, and multi-textures are extracted based on local landform from shallow-to-deep feature learning. After the network being trained on scene-level samples and guided by the superpixel segmentation, Martian landforms can be correctly classified in an efficient way, whose mapping time cost sharply decreased when compared to the reference methods. The proposed MarsMapNet has been validated on three real landing sites from several Mars missions (i.e., the Jezero Crater, the Southern Utopia Planitia, the Oxia Planum) by using the Mars Reconnaissance Orbiter's Context Camera (CTX) images. Qualitative and quantitative analysis on the obtained experimental results confirm the effectiveness and efficiency of the proposed MarsMapNet when compared with the state-of-theart methods, demonstrating its potential for supporting a Martian global landform mapping in the future.

show abstract

Mars Image Content Classification: Three Years of NASA Deployment and Recent Advances

Cited by 18 publications

References 14 publications

Interactive Mars Image Content-Based Search with Interpretable Machine Learning

Interactive Mars Image Content-Based Search with Interpretable Machine Learning

LROCNet: Detecting Impact Ejecta and Older Craters on the Lunar Surface

MarsMapNet: A Novel Superpixel-Guided Multiview Feature Fusion Network for Efficient Martian Landform Mapping

Contact Info

Product

Resources

About