CloudFindr: A Deep Learning Cloud Artifact Masker for Satellite DEM Data

Abstract

Artifact removal is an integral component of cinematic scientific visualization, and is especially challenging with big datasets in which artifacts are difficult to define. In this paper, we describe a method for creating cloud artifact masks which can be used to remove artifacts from satellite imagery using a combination of traditional image processing together with deep learning based on U-Net. Compared to previous methods, our approach does not require multi-channel spectral imagery but performs successfully on single-channel Digital Elevation Models (DEMs). DEMs are a representation of the topography of the Earth and have a variety applications including planetary science, geology, flood modeling, and city planning.

DL4SciVis: A State-of-the-Art Survey on Deep Learning for Scientific Visualization

“…This makes unsupervised learning, particularly self-supervised learning, a suitable candidate for accomplishing such tasks. Thus, investigating the underexplored self-supervised learning solutions for making predictions or recommendations will certainly boost [2], [27], [45], [46], [47], [49], [50] [12], [63] [35], [57] [11], [15] [24] + [51], [53], [54], [55] , [56], [76], [84] [70], [158] [71], [85] [31], [40] [126], [131], [156] + , [160], [162] [159]…”

“…We single out seven works (i.e., [26], [27], [84], [162], [163], [164], [168]) and label them in the category of fluid simulation, as the primary focus of these works is simulation. Finally, two works (i.e., [97], [103]) target particle data, and the remaining two (i.e., [15], [183]) deal with image data. [18] isosurface similarity Jaccard Jaccard index in binary classification (visual object class) LPIPS [175] learned perceptual image patch similarity LSiM [89] 1.…”

“…Their MLP-based model learns particle end locations given their start locations and file cycles. [154] VC-Net 3D volume patch, multislice composited 2D MIP vessel mask Nguyen et al [116] cryo-EM image, dense pseudo labels soft labels Ghahremani et al [40] NeuroConstruct batch of grayscale images probability map He et al [65] super-voxel graph with neighborhood relations feature classification per super-voxel Deng et al [31] Vortex-Net sample local patch hard labels Berenjkoub et al [11] velocity patch binary classification of vortex boundary Kashir et al [81] input map (velocity, vorticity) binary segmentation of vortical structure Borkiewicz et al [15] CloudFindr image patch predicted mask…”

“…In addition, they investigated the symmetric Ushaped network structure to find the optimal settings to best extract vortical structures. Borkiewicz et al [15] developed CloudFindr to detect cloud from satellite image data. Their U-Net-based solution produces a predicted mask given the input image patch.…”

“…A close look at the categorization under CNN shows that encoder+decoder is the most diverse, covering all five research tasks, while decoder covers two tasks (i.e., data [2](U-Net, deformable CNN), [47] [11](CNN vs. ResNet vs. U-Net), [15](U-Net), [27] (GAN) [49] [50](GAN), [55] (GAN) [35], [40](nested encoder+decoder), [45], [46], [53] [76](ESPCN), [106] (GAN), [126] [54] (GAN), [56] (GAN), [70] (GAN) [159] (FRVSR-Net), [161](EnhanceNet) [81](symmetric FCN), [84](AE), [103](residual AE) [162] (multi-pass GAN) [116](U-Net vs. U-Net+ResNet vs. DenseNet) [167] (ESRGAN, WGAN) [117](AE), [154](multi-stream CNN) [168] (GAN), [182] [158](U-Net, V-Net), [163], [164] MLP [57], [183](FCCNN) - [84], [109], [160] RNN [164](stacked LSTM) [163](LSTM) -GNN [52](GCN), [64](GCN) [131](GCN) -CNN+MLP [24], [26](Siamese) [63] [12] (GAN), [48](AE) [31], [62], [85] [97](Geo-CNN)…”