LGHAP: the Long-term Gap-free High-resolution Air Pollutant concentration dataset, derived via tensor-flow-based multimodal data fusion

Abstract

Abstract. Developing a big data analytics framework for generating the Long-term Gap-free High-resolution Air Pollutant concentration dataset (abbreviated as LGHAP) is of great significance for environmental management and Earth system science analysis. By synergistically integrating multimodal aerosol data acquired from diverse sources via a tensor-flow-based data fusion method, a gap-free aerosol optical depth (AOD) dataset with a daily 1 km resolution covering the period of 2000–2020 in China was generated. Specifically, data gaps in daily AOD imageries from the Moderate Resolution Imaging Spectroradiometer (MODIS) aboard Terra were reconstructed based on a set of AOD data tensors acquired from diverse satellites, numerical analysis, and in situ air quality measurements via integrative efforts of spatial pattern recognition for high-dimensional gridded image analysis and knowledge transfer in statistical data mining. To our knowledge, this is the first long-term gap-free high-resolution AOD dataset in China, from which spatially contiguous PM2.5 and PM10 concentrations were then estimated using an ensemble learning approach. Ground validation results indicate that the LGHAP AOD data are in good agreement with in situ AOD observations from the Aerosol Robotic Network (AERONET), with an R of 0.91 and RMSE equaling 0.21. Meanwhile, PM2.5 and PM10 estimations also agreed well with ground measurements, with R values of 0.95 and 0.94 and RMSEs of 12.03 and 19.56 µg m−3, respectively. The LGHAP provides a suite of long-term gap-free gridded maps with a high resolution to better examine aerosol changes in China over the past 2 decades, from which three major variation periods of haze pollution in China were revealed. Additionally, the proportion of the population exposed to unhealthy PM2.5 increased from 50.60 % in 2000 to 63.81 % in 2014 across China, which was then reduced drastically to 34.03 % in 2020. Overall, the generated LGHAP dataset has great potential to trigger multidisciplinary applications in Earth observations, climate change, public health, ecosystem assessment, and environmental management. The daily resolution AOD, PM2.5, and PM10 datasets are publicly available at https://doi.org/10.5281/zenodo.5652257 (Bai et al., 2021a), https://doi.org/10.5281/zenodo.5652265 (Bai et al., 2021b), and https://doi.org/10.5281/zenodo.5652263 (Bai et al., 2021c), respectively. Monthly and annual datasets can be acquired from https://doi.org/10.5281/zenodo.5655797 (Bai et al., 2021d) and https://doi.org/10.5281/zenodo.5655807 (Bai et al., 2021e), respectively. Python, MATLAB, R, and IDL codes are also provided to help users read and visualize these data.

Reconstructing 6-hourly PM2.5 datasets from 1960 to 2020 in China

“…Table 1 compares our dataset and the available datasets in primary predictors, temporal resolution, and CV results (Ma et al, 2016;Fang et al, 2016;Liu et al, 2017;Xiao et al, 2018;Xue et al, 2019;Liang et al, 2020;Huang et al, 2021;Wei et al, 2021a;Van Donkelaar et al, 2021;Geng et al, 2021;Bai et al, 2022). AOD-based datasets are only available from around 2000 at the earliest, with temporal resolutions ranging from daily scale to monthly scale.…”

“…The multi-year trend of our gridded PM 2.5 dataset is also compared with those of publicly available datasets, including the TAP data (Geng et al, 2021), the GEFPM data (Van Donkelaar et al, 2021), the LGHAP data (Bai et al, 2022), and the CHAP data (Wei et al, 2021a), which have been interpolated to the same grid resolution. Figure 9 shows the spatial distributions of PM 2.5 from those datasets at 5-year intervals between 2000-2020.…”

“…Increased PM 2.5 has strongly interacted with solar radiation through absorption and scattering, thereby reducing visibility and influencing the Earth's radiance balance. Inhalable PM 2.5 has increased human morbidity and mortality by penetrating the respiratory system (Pope et al, 2002;Beelen et al, 2007;. To evaluate the impacts of PM 2.5 pollution on the environment, climate, and health, the primary concern is to understand the spatiotemporal variations of PM 2.5 concentrations.…”

Spatiotemporal continuous estimates of daily 1 km PM2.5 from 2000 to present under the Tracking Air Pollution in China (TAP) framework

“…Xiao et al: Spatiotemporal continuous daily 1 km PM 2.5 estimation in China For example, moderate-resolution imaging spectroradiometer (MODIS) products provide aerosol optical depth (AOD) retrievals at a 3 km resolution, contributing to the prediction of ground-level PM 2.5 concentrations at the local scale (Xie et al, 2015;He and Huang, 2018;. The multiangle implementation of atmospheric correction (MA-IAC) algorithm provides AOD retrievals at a 1 km resolution and benefits predictions of PM 2.5 distribution at a 1 km (Wei et al, 2021;Goldberg et al, 2019;Xiao et al, 2017;Bai et al, 2022b) or higher spatial resolution . Recently, with the Gaofen-5 satellite retrieval, Zhang et al (2018) predicted the PM 2.5 concentration at a 160 m resolution.…”

“…However, most of these high-resolution data products are limited to after 2013 or cover a specific region of China. Few studies have filled the missing predictions that have resulted from missing satellite retrievals (Bai et al, 2022b;Ma et al, 2022). This discontinuous PM 2.5 prediction in space and time not only limits the application of PM 2.5 products in scientific research and environmental management but also biases the characterization of population exposure to PM 2.5 pollution (Xiao et al, 2017).…”