Aerosol and Cloud Detection Using Machine Learning Algorithms and Space-Based Lidar Data

Abstract

Clouds and aerosols play a significant role in determining the overall atmospheric radiation budget, yet remain a key uncertainty in understanding and predicting the future climate system. In addition to their impact on the Earth’s climate system, aerosols from volcanic eruptions, wildfires, man-made pollution events and dust storms are hazardous to aviation safety and human health. Space-based lidar systems provide critical information about the vertical distributions of clouds and aerosols that greatly improve our understanding of the climate system. However, daytime data from backscatter lidars, such as the Cloud-Aerosol Transport System (CATS) on the International Space Station (ISS), must be averaged during science processing at the expense of spatial resolution to obtain sufficient signal-to-noise ratio (SNR) for accurately detecting atmospheric features. For example, 50% of all atmospheric features reported in daytime operational CATS data products require averaging to 60 km for detection. Furthermore, the single-wavelength nature of the CATS primary operation mode makes accurately typing these features challenging in complex scenes. This paper presents machine learning (ML) techniques that, when applied to CATS data, (1) increased the 1064 nm SNR by 75%, (2) increased the number of layers detected (any resolution) by 30%, and (3) enabled detection of 40% more atmospheric features during daytime operations at a horizontal resolution of 5 km compared to the 60 km horizontal resolution often required for daytime CATS operational data products. A Convolutional Neural Network (CNN) trained using CATS standard data products also demonstrated the potential for improved cloud-aerosol discrimination compared to the operational CATS algorithms for cloud edges and complex near-surface scenes during daytime.

Planetary Boundary Layer Height Estimates From ICESat-2 and CATS Backscatter Measurements

“…One month of data was chosen because it was large enough to contain a representative sampling of PBL cloud and aerosol scenes that are representative of the PBLH, yet small enough to train the CNN relatively quickly. Previously published studies that trained CNN models suggest using a training dataset that is ∼20% of your prediction dataset provides accurate predictions (Shahin et al, 2004;Yorks et al, 2021). Supervised machine learning algorithms, such as CNN, have been utilized for feature recognition and object detection in images for years (Gidaris and Komodakis, 2015).…”

“…The CNN architecture and training method used in this study to estimate PBLH were similar to what was defined in Yorks et al (2021). However, there were some key differences.…”

“…Areas where the lidar signal was attenuated, or the threshold approach did not determine a PBL height were filtered out. The other difference between the methodology in Yorks et al (2021) is that here binary feature classification was used as opposed to multiple feature classification; however, this only implies an adjustment to the output layer of the CNN. The training dataset of PBLH defined by the threshold technique had horizontal resolutions of 7 km (night) and 80 km (day), different than the resolutions described in Threshold Method.…”

“…This SNR results in low minimum detectable backscatter (MDB), the lowest backscatter value that PBL clouds and the aerosol mixing height can be detected (Yorks et al, 2016). Additionally, the total backscatter signal at 1,064 nm has a smaller molecular contribution, especially near the surface, compared to longer wavelengths (Yorks et al, 2021). CATS also observed different local times each overpass, covering the full diurnal cycle roughly every 60 days due to the 51.8°inclination angle of the ISS orbit.…”

Laser Beam Atmospheric Propagation Modelling for Aerospace LIDAR Applications

“…Interestingly, t-SNE defines bad profiles as small groups; hence, it distinguishes high background counts as a smaller group that is finally labelled as noise; this is how t-SNE helps to increase SNR. Yorks et al [107] also worked in improving SNR of backscatter LIDARs in daytime and discriminating cloud and aerosol. In order to proceed with that, this study considers an ML approach, convolutional neural network (CNN), as it is a time-efficient and widely used approach.…”