Barefoot Rover: a Sensor-Embedded Rover Wheel Demonstrating In-Situ Engineering and Science Extractions using Machine Learning

TCN: Pioneering Topological-Based Convolutional Networks for Planetary Terrain Learning

“…For example, Wagstaff et al (2018) use a neural network architecture based on an autoencoder to capture and explain novel features in multispectral images. Additionally, Marchetti et al (2020) utilize tree-based Stochastic Gradient Boosting (SGB) to extract information from in-situ sensors and train models for terrain type classification and slip regression. Kerner et al (2020) compare the performance of four detection methods and detect novel geology on multispectral images from planetary instrument datasets.…”

“…For instance, Figure 1(b) shows an example of experiment setup where the Barefoot Rover mobility cart with the tactile wheel mounted on it sits in a metal trough over regolith with letters Jet Propulsion Laboratory (JPL) spelled in small rocks on the surface. Background and details for the Barefoot project, the dataset and data processing can be obtained in Marchetti et al (2020) and Chen, Marchetti, and Gel (2021). Figure 1: An example image from the Barefoot surface pressure dataset (Lightholder et al 2021).…”

“…Most of the pressure images have calibrated pressure at around 0, since at any given time the wheel is only experiencing pressure over a very small area, while the rest of its circumference is not in contact with the ground. Marchetti et al (2020) collected over a thousand experiments across different materials, terrain patterns and slippage values. We use a subset of the experiments (Lightholder et al 2021) for surface patterns, including various rock types, to classify pressure sensor images into eight classes: bedrock, flat, gullies, pebbles, rock-above, rock-below, sharpdunes, smoodunes.…”

“…Our project presents an early-concept approach which aims to make a step in bridging the power of DL with onboard exploration of planetary terrain, by substantially reducing computational and storage costs, while maintaining the high classification accuracy. In particular, compared to Marchetti et al (2020), in our model, we use the low resolution surface pattern image for image representation learning and representation learning based on local topological information, i.e., using only 6.25% image information for multi-label classification tasks. Another benefit of TCN is that the topological signatures can be computed offline from images, which substantially reduces online computation time necessary.…”

Deepening the Sense of Touch in Planetary Exploration with Geometric and Topological Deep Learning

“…A new concept developed in Marchetti et al (2020) not only incorporates in-situ sensors on a robotic wheel, but also uses machine learning to extract meaningful metrics from the interaction between the wheel and the terrain using the sensors. These include continuous slip estimation, balance, and sharpness for engineering purposes and estimates of hydration, texture, and terrain patterns for science applications.…”