Benchmarking Deep Learning Models for Cloud Detection in Landsat-8 and Sentinel-2 Images

López-Puigdollers, Dan; Mateo-García, Gonzalo; Gómez-Chova, Luis

doi:10.3390/rs13050992

Cited by 55 publications

(36 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The first subsection provides Kappa-Mask L1C and L2A comparison with rule-based methods-Sen2Cor, Fmask and MAJA. Afterwards, KappaMask performance is compared to machine learning methods-S2cloudless that uses tree algorithm with LightGBM and deep learning DL_L8S2_UV [21]…”

Section: Resultsmentioning

confidence: 99%

“…The first subsection provides KappaMask L1C and L2A comparison with rule-based methods-Sen2Cor, Fmask and MAJA. Afterwards, KappaMask performance is compared to machine learning methods-S2cloudless that uses tree algorithm with LightGBM and deep learning DL_L8S2_UV [21] network. The next sub-section shows the feature importance analysis of KappaMask L1C and L2A model, followed by the hyperparameter tuning of network depth and number of filters.…”

Section: Resultsmentioning

confidence: 99%

“…The approach leverages advantages of the 20 m resolution dataset and increases its robustness to different landscapes. The evaluation on the test dataset is shown, including comparison with rule-based methods-Sen2Cor, Fmask, MAJA-and AI-based methods-S2cloudless, DL-L8S2-UV (Deep Learning for Cloud Detection in Landsat-8 and Sentinel-2 Images [21]). The feature importance in the model for both L1C and L2A input is analysed.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

KappaMask: AI-Based Cloudmask Processor for Sentinel-2

Domnich

Sünter²,

Trofimov³

et al. 2021

Remote Sensing

View full text Add to dashboard Cite

The Copernicus Sentinel-2 mission operated by the European Space Agency (ESA) provides comprehensive and continuous multi-spectral observations of all the Earth’s land surface since mid-2015. Clouds and cloud shadows significantly decrease the usability of optical satellite data, especially in agricultural applications; therefore, an accurate and reliable cloud mask is mandatory for effective EO optical data exploitation. During the last few years, image segmentation techniques have developed rapidly with the exploitation of neural network capabilities. With this perspective, the KappaMask processor using U-Net architecture was developed with the ability to generate a classification mask over northern latitudes into the following classes: clear, cloud shadow, semi-transparent cloud (thin clouds), cloud and invalid. For training, a Sentinel-2 dataset covering the Northern European terrestrial area was labelled. KappaMask provides a 10 m classification mask for Sentinel-2 Level-2A (L2A) and Level-1C (L1C) products. The total dice coefficient on the test dataset, which was not seen by the model at any stage, was 80% for KappaMask L2A and 76% for KappaMask L1C for clear, cloud shadow, semi-transparent and cloud classes. A comparison with rule-based cloud mask methods was then performed on the same test dataset, where Sen2Cor reached 59% dice coefficient for clear, cloud shadow, semi-transparent and cloud classes, Fmask reached 61% for clear, cloud shadow and cloud classes and Maja reached 51% for clear and cloud classes. The closest machine learning open-source cloud classification mask, S2cloudless, had a 63% dice coefficient providing only cloud and clear classes, while KappaMask L2A, with a more complex classification schema, outperformed S2cloudless by 17%.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

KappaMask: AI-Based Cloudmask Processor for Sentinel-2

Domnich

Sünter²,

Trofimov³

et al. 2021

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…The cloud mask accuracy of s2cloudless is reported to be on par with the multi-temporal classifier MAJA [25], but the considered detector can be applied on mono-temporal satellite observations. Note that, alternatively to s2cloudless, the masks m may be computed via a dedicated neural network for cloud detection [26], [27]. However, s2cloudless has proven to be lightweight and provide sufficient performance at little extra computational cost in run time or memory, making it an appealing cloud detector to be applied on a large-scale date set such as SEN12MS-CR-TS.…”

Section: Cloud Detection and Mask Computationmentioning

confidence: 99%

SEN12MS-CR-TS: A Remote-Sensing Data Set for Multimodal Multitemporal Cloud Removal

Ebel

Schmitt

et al. 2022

IEEE Trans. Geosci. Remote Sensing

View full text Add to dashboard Cite

About half of all optical observations collected via spaceborne satellites are affected by haze or clouds. Consequently, cloud coverage affects the remote sensing practitioner's capabilities of a continuous and seamless monitoring of our planet. This work addresses the challenge of optical satellite image reconstruction and cloud removal by proposing a novel multi-modal and multi-temporal data set called SEN12MS-CR-TS. We propose two models highlighting the benefits and use cases of SEN12MS-CR-TS: First, a multi-modal multi-temporal 3D-Convolution Neural Network that predicts a cloud-free image from a sequence of cloudy optical and radar images. Second, a sequence-to-sequence translation model that predicts a cloud-free time series from a cloud-covered time series. Both approaches are evaluated experimentally, with their respective models trained and tested on SEN12MS-CR-TS. The conducted experiments highlight the contribution of our data set to the remote sensing community as well as the benefits of multi-modal and multitemporal information to reconstruct noisy information. Our data set is available at https://patrickTUM.github.io/cloud removal.

show abstract

“…We shared the python code at the following location: https://github.com/sentinelhub/sentinel2-cloud-detector.git. The algorithm is now widely popular and has become one of the latest algorithms used for cloud detection(Dan et al 2021). In this study, any pixel that is agged by s2cloudless as a medium or high con dence cloud, cloud shadow, or cirrus cloud is discarded.Both Landsat-7 Collection 1 SR and Landsat-8 Collection 1 SR products include a pixel quality band(Foga et al 2017).…”

mentioning

confidence: 99%

Harmonizing Surface Reflectance Between Landsat-7 ETM+, Landsat-8 OLI, and Sentinel-2 MSI

Cao

Han

Liangzhi

2021

Preprint

View full text Add to dashboard Cite

Remote sensing dynamic monitoring methods often benefit from a dense time series of observations. To enhance these time series, it is sometimes necessary to integrate data from multiple satellite systems. For more than 40 years, Landsat has provided the longest time record of space-based land surface observations, and the successful launch of the Landsat-8 Operational Land Imager (OLI) sensor in 2013 continues this tradition. However, the 16-day observation period of Landsat images has challenged the ability to measure subtle and transient changes like never before. The European Space Agency (ESA) launched the Sentinel-2A satellite in 2015. The satellite carries a Multispectral Instrument (MSI) sensor that provides a 10-20m spatial resolution data source providing an opportunity to complement the Landsat data record. The collection of Sentinel-2A MSI, Landsat-7 ETM+, and Landsat-8 OLI data provide multispectral global coverage from 10m to 30m with further reduced data revisit intervals. There are many differences between sensor data that need to be taken into account to use these data together reliably. The purpose of this study is to evaluate the potential of integrating surface reflectance data from Landsat-7, Landsat-8 and Sentinel-2 archived in the Google Earth Engine (GEE) cloud platform. To test and quantify the differences between these sensors, hundreds of thousands of surface reflectance data from sensor pairs were collected over China. In this study, some differences in the surface reflectance of the sensor pairs were identified, based upon which a cross-sensor conversion model was proposed, i.e., a suitable adjustment equation was fitted using an ordinary least squares (OLS) linear regression method to convert the Sentinel-2 reflectance values closer to the Landsat-7 or Landsat-8 values. The regression results show that the Sentinel MSI data are spectrally comparable to both types of Landsat image data, just as the Landsat sensors are comparable to each other. The root mean square error (RMSE) values between MSI and Landsat spectral values before coordinating the sensors ranged from 0.014 to 0.037, and the RMSE values between OLI and ETM + ranged from 0.019 to 0.039. After coordination, RMSE values between MSI and Landsat spectral values ranged from 0.011 to 0.026, and RMSD values between OLI and ETM + ranged from 0.013 to 0.034. The fitted adjustment equations were also compared to the HLS (Harmonized Landsat-8 Sentinel-2) global fitted equations (Sentinel-2 to Landsat-8) published by the National Aeronautics and Space Administration (NASA) and were found to be significantly different, increasing the likelihood that such adjustments would need to be fitted on a regional basis. This study believes that despite the differences in these datasets, it appears feasible to integrate these datasets by applying a linear regression correction between the bands.

show abstract

Benchmarking Deep Learning Models for Cloud Detection in Landsat-8 and Sentinel-2 Images

Cited by 55 publications

References 39 publications

KappaMask: AI-Based Cloudmask Processor for Sentinel-2

KappaMask: AI-Based Cloudmask Processor for Sentinel-2

SEN12MS-CR-TS: A Remote-Sensing Data Set for Multimodal Multitemporal Cloud Removal

Harmonizing Surface Reflectance Between Landsat-7 ETM+, Landsat-8 OLI, and Sentinel-2 MSI

Contact Info

Product

Resources

About