Next-Generation Optical Sensing Technologies for Exploring Ocean Worlds—NASA FluidCam, MiDAR, and NeMO-Net

Abstract

We highlight three emerging NASA optical technologies that enhance our ability to remotely sense, analyze, and explore ocean worlds-FluidCam and fluid lensing, MiDAR, and NeMO-Net. Fluid lensing is the first remote sensing technology capable of imaging through ocean waves without distortions in 3D at sub-cm resolutions. Fluid lensing and the purpose-built FluidCam CubeSat instruments have been used to provide refraction-corrected 3D multispectral imagery of shallow marine systems from unmanned aerial vehicles (UAVs). Results from repeat 2013 and 2016 airborne fluid lensing campaigns over coral reefs in American Samoa present a promising new tool for monitoring fine-scale ecological dynamics in shallow aquatic systems tens of square kilometers in area. MiDAR is a recently-patented active multispectral remote sensing and optical communications instrument which evolved from FluidCam. MiDAR is being tested on UAVs and autonomous underwater vehicles (AUVs) to remotely sense living and non-living structures in light-limited and analog planetary science environments. MiDAR illuminates targets with high-intensity narrowband structured optical radiation to measure an object's spectral reflectance while simultaneously transmitting data. MiDAR is capable of remotely sensing reflectance at fine spatial and temporal scales, with a signal-to-noise ratio 10-10 3 times higher than passive airborne and spaceborne remote sensing systems, enabling high-framerate multispectral sensing across the ultraviolet, visible, and near-infrared spectrum. Preliminary results from a 2018 mission to Guam show encouraging applications of MiDAR to imaging coral from airborne and underwater platforms whilst transmitting data across the air-water interface. Finally, we share NeMO-Net, the Neural Multi-Modal Observation & Training Network for Global Coral Reef Assessment. NeMO-Net is a machine learning technology under development that exploits high-resolution data from FluidCam and MiDAR for augmentation of low-resolution airborne and satellite remote sensing. NeMO-Net is intended to harmonize the growing diversity of 2D and 3D remote sensing with in situ data into a single open-source platform for assessing shallow marine ecosystems globally Chirayath and Li NASA FluidCam, MiDAR, and NeMO-Net using active learning for citizen-science based training. Preliminary results from four-class coral classification have an accuracy of 94.4%. Together, these maturing technologies present promising scalable, practical, and cost-efficient innovations that address current observational and technological challenges in optical sensing of marine systems.

Going Batty: The Challenges and Opportunities of Using Drones to Monitor the Behaviour and Habitat Use of Rays

“…However, we advise caution when considering purchasing these systems, and suggest ensuring that the networks required to run the corrections are available in operator areas of interest. Finally, we look forward to further advances in machine learning as a means to accurately identify and automate the mapping of appropriate ray habitat [73,[76][77][78]. With the large volumes of data captured by drones, they are ideally suited to building artificial intelligence models, so they present an opportunity as much as a challenge.…”

Structure-from-Motion on shallow reefs and beaches: potential and limitations of consumer-grade drones to reconstruct topography and bathymetry

“…Using airborne images to analyze underwater environments requires (a) calm water surfaces, (b) is dependent on the Secchi depth (visibility by means of turbidity) and (c) is dependent on surface textures of underwater areas (Mandlburger 2019). Within the water column, light is continuously attenuated through absorption and scattering along the optical path (Chirayath and Earle 2016;Chirayath and Li 2019). Bathymetric reconstruction in calmer and deeper waters with user grade UAVs (and without LiDAR sensors) is possible, but is characterized by large uncertainties and becomes more challenging with increasing water depths and hydrodynamic activity (Casella et al 2017).…”

Towards Benthic Habitat 3D Mapping Using Machine Learning Algorithms and Structures from Motion Photogrammetry

“…Still, these results should encourage further studies on the proposed approach, such as using ROV systems to produce higher-quality images for monitoring deep seafloor areas [30]. Moreover, studying the same proposed approaches with the new Fluid lensing optical multispectral instrument (FluidCam) and Multispectral Imaging Detection and Active Reflectance instrument (MiDAR) developed by NASA [62]. FluidCam can produce 3D multispectral images corrected from refraction for shallow marine environments.…”