New Gap-Filling Strategies for Long-Period Flux Data Gaps Using a Data-Driven Approach

Kang, Minseok; Ichii, Kazuhito; Kim, Joon; Indrawati, Yohana Maria; Park, Juhan; Moon, Minkyu; Lim, Jong-Hwan; Chun, Jung-Hwa

doi:10.3390/atmos10100568

Cited by 28 publications

(24 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In particular, Kang et al 2012 showed that the gap-filled ET data, derived by using MDS under wet canopy conditions, were underestimated because the data used in the gap-filling methods were mostly collected during dry or partially wet canopy conditions; MDS also failed in regard to the consideration of the aerodynamic coupling, advection of sensible heat, and heat storage. Because MDS performed poorly for long-period flux data gaps i.e., gaps longer than a month because of the absence of marginally distributed data around gaps, Kang et al 2019a suggested that researchers apply a data-driven approach using machine learning and remote-sensing data to apply gap-filling for the long gaps.…”

Section: Data Processingmentioning

confidence: 99%

Progress in water and energy flux studies in Asia: A review focused on eddy covariance measurements

Kang¹,

Cho

2021

J. Agric. Meteorol.

View full text Add to dashboard Cite

The eddy covariance EC technique-based observation system allows for researchers to determine latent and sensible heat fluxes, which are key components of the surface energy balance. The number of water and energy flux studies in Asia has increased as the number of flux measurement sites and the length of the observation periods have grown. To retrace the footprints of the AsiaFlux network and predict future research directions, we reviewed the progress in water and energy flux studies in Asia from the 1990s to the present day. This included studies on continuous evapotranspiration ET and surface energy balance measurements in various ecosystems, from the tropics to the polar regions. We also reviewed comparative experiments between the EC technique and other observation techniques including the use of a lysimeter or scintillometer, data processing techniques, connections between carbon and water fluxes, and multi-site syntheses. This paper discusses three remaining challenges that are hindering the derivation of scientific knowledge for ET and the surface energy balance, namely: the non-closure of the surface energy budget, imperfect compatibility between open-and closed-path gas analyzers, and difficulty in partitioning ET into evaporation and transpiration. If we leverage the advantages of the EC technique i.e., high sampling rates of ≥ 10 Hz and continuous measurement capabilities , standardized methods for correcting and partitioning can be developed in the near future.

show abstract

Section: Data Processingmentioning

confidence: 99%

Progress in water and energy flux studies in Asia: A review focused on eddy covariance measurements

Kang¹,

Cho

2021

J. Agric. Meteorol.

View full text Add to dashboard Cite

show abstract

“…This kind of EML would be useful for gap filling or the evaluation of GPP time series of the site that generated the model. Machine learning algorithms can fill gaps longer than 30 d (Kang et al, 2019).…”

Section: Agreementmentioning

confidence: 99%

Machine learning estimates of eddy covariance carbon flux in a scrub in the Mexican highland

et al. 2021

View full text Add to dashboard Cite

Abstract. Arid and semiarid ecosystems contain relatively high species diversity and are subject to intense use, in particular extensive cattle grazing, which has favored the expansion and encroachment of perennial thorny shrubs into the grasslands, thus decreasing the value of the rangeland. However, these environments have been shown to positively impact global carbon dynamics. Machine learning and remote sensing have enhanced our knowledge about carbon dynamics, but they need to be further developed and adapted to particular analysis. We measured the net ecosystem exchange (NEE) of C with the eddy covariance (EC) method and estimated gross primary production (GPP) in a thorny scrub at Bernal in Mexico. We tested the agreement between EC estimates and remotely sensed GPP estimates from the Moderate Resolution Imaging Spectroradiometer (MODIS), and also with two alternative modeling methods: ordinary-least-squares (OLS) regression and ensembles of machine learning algorithms (EMLs). The variables used as predictors were MODIS spectral bands, vegetation indices and products, and gridded environmental variables. The Bernal site was a carbon sink even though it was overgrazed, the average NEE during 15 months of 2017 and 2018 was −0.78 gCm-2d-1, and the flux was negative or neutral during the measured months. The probability of agreement (θs) represented the agreement between observed and estimated values of GPP across the range of measurement. According to the mean value of θs, agreement was higher for the EML (0.6) followed by OLS (0.5) and then MODIS (0.24). This graphic metric was more informative than r2 (0.98, 0.67, 0.58, respectively) to evaluate the model performance. This was particularly true for MODIS because the maximum θs of 4.3 was for measurements of 0.8 gCm-2d-1 and then decreased steadily below 1 θs for measurements above 6.5 gCm-2d-1 for this scrub vegetation. In the case of EML and OLS, the θs was stable across the range of measurement. We used an EML for the Ameriflux site US-SRM, which is similar in vegetation and climate, to predict GPP at Bernal, but θs was low (0.16), indicating the local specificity of this model. Although cacti were an important component of the vegetation, the nighttime flux was characterized by positive NEE, suggesting that the photosynthetic dark-cycle flux of cacti was lower than ecosystem respiration. The discrepancy between MODIS and EC GPP estimates stresses the need to understand the limitations of both methods.

show abstract

“…Gap-lling of continuous water quality datasets is a so far unexplored, yet potentially powerful application of Machine Learning. Machine Learning algorithms have been successfully applied for gap-lling in medical datasets (Shah et al, 2014), eddy-covariance evaporation and CO 2 ux data sets (Kang et al, 2019), soil moisture (Mao et al, 2019) and more recently also for daily stream ow time series (Arriagada et al, 2021) The objectives of this study were to: (i) to evaluate six different Machine Learning models for gap-lling in a high-frequency NO 3 and TP concentration time series, (ii) to showcase the potential added value and limitations of Machine Learning to interpret underlying nutrient transport processes, and (iii) to study the limits of Machine Learning algorithms for making predictions outside the training period. As case study, we used four years of high-frequency data from a ditch draining one dairy farm in the east of The Netherlands where the nutrient transport from the soil to the surface water were previously investigated (Barcala et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

Value and limitations of Machine Learning in high-frequency nutrient data for gap- filling, forecasting, and transport process interpretation

Barcala

Rozemeijer

Ouwerkerk

et al. 2022

Preprint

View full text Add to dashboard Cite

High-frequency monitoring of water quality in catchments brings along the challenge of post-processing large amounts of data. Moreover, monitoring stations are often remote and technical issues resulting in data gaps are common. Machine Learning algorithms can be applied to fill these gaps, and to a certain extent, for predictions and interpretation. The objectives of this study were (1) to evaluate six different Machine Learning models for gap-filling in a high-frequency nitrate and total-phosphorus concentration time series, (2) to showcase the potential added value (and limitations) of Machine Learning to interpret underlying processes, and (3) to study the limits of Machine Learning algorithms for predictions outside the training period. We used a four-year high-frequency dataset from a ditch draining one intensive dairy farm in the east of The Netherlands. Continuous time series of precipitation, evaporation, groundwater levels, discharge, turbidity, and nitrate or total-phosphorus were used as predictors for total-phosphorus and nitrate concentrations respectively. Our results showed that the Random Forest algorithm had the best performance to fill in data-gaps, with R2 higher than 0.92 and short computation times. The feature importance helped understanding the changes in transport processes linked to water conservation measures and rain variability. Applying the Machine Learning model outside the training period resulted in a low performance, largely due to system changes (manure surplus and water conservation) which were not included as predictors. This study offers a valuable and novel example of how to use and interpret Machine Learning models for post-processing high-frequency water quality data.

show abstract

New Gap-Filling Strategies for Long-Period Flux Data Gaps Using a Data-Driven Approach

Cited by 28 publications

References 34 publications

Progress in water and energy flux studies in Asia: A review focused on eddy covariance measurements

Progress in water and energy flux studies in Asia: A review focused on eddy covariance measurements

Machine learning estimates of eddy covariance carbon flux in a scrub in the Mexican highland

Value and limitations of Machine Learning in high-frequency nutrient data for gap- filling, forecasting, and transport process interpretation

Contact Info

Product

Resources

About