Comparison of discrete Fourier transform (DFT) and principal component analysis/DFT as forecasting tools for absorbance time series received by UV-visible probes installed in urban sewer systems

Plazas-Nossa, Leonardo; Torres, Andrés

doi:10.2166/wst.2014.011

Cited by 9 publications

(7 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…According to [26], SPCA is more efficient than a basic PCA, due to the fact that the power spectrum is invariant to delays and phase shifts, and that SPCA is less sensitive to the range of missing data (by comparison the range of existing ones) than PCA. Plazas-Nossa and Torres [27] have compared two methods: one based on Discrete FT (DFT), and one mixed with PCA (PCA/DFT). According to the NRMSD criteria ( be used with two moving windows: the Hamming or the Kaiser-Bessel [11].…”

Section: Methods Based On Fourier's Theory: Use Of Signal Analysis Mementioning

confidence: 99%

See 1 more Smart Citation

Interpolation in Time Series: An Introductive Overview of Existing Methods, Their Performance Criteria and Uncertainty Assessment

Lepot

Aubin

Clemens

2017

Water

203

View full text Add to dashboard Cite

Abstract:A thorough review has been performed on interpolation methods to fill gaps in time-series, efficiency criteria, and uncertainty quantifications. On one hand, there are numerous available methods: interpolation, regression, autoregressive, machine learning methods, etc. On the other hand, there are many methods and criteria to estimate efficiencies of these methods, but uncertainties on the interpolated values are rarely calculated. Furthermore, while they are estimated according to standard methods, the prediction uncertainty is not taken into account: a discussion is thus presented on the uncertainty estimation of interpolated/extrapolated data. Finally, some suggestions for further research and a new method are proposed.

show abstract

Section: Methods Based On Fourier's Theory: Use Of Signal Analysis Mementioning

confidence: 99%

“…WT seems to be useful for data with sudden changes [25]. Each of these transforms based on Fourier's theory (FT, DFT, CFT, STFT, DWT, CWT) can be reversed to assess X(t) and fill in gaps: see e.g., [29] for CWT or [27] for DFT.…”

Section: Methods Based On Fourier's Theory: Use Of Signal Analysis Mementioning

confidence: 99%

Interpolation in Time Series: An Introductive Overview of Existing Methods, Their Performance Criteria and Uncertainty Assessment

Lepot

Aubin

Clemens

2017

Water

203

View full text Add to dashboard Cite

show abstract

“…Several value combinations of r and m parameters were verified, and the best results using r and m proved to be a value of ten (10). Although (30,50 or 70) for r and m parameters translate into less outlier values, the time series' shape becomes stair-like, losing its similarity to the shape of the original times series (Figure 3, graph a). Thus, using these large window size values would be disadvantageous because an analysis of the last part of the time series (roughly values 30950 until 31000) becomes largely impossible, as observed in Figure 3 (graphs d, e and f).…”

Section: Resultsmentioning

confidence: 99%

“…This movement represents a return to the time domain (as opposed to the time to frequency shift found in the initial DFT process), as shown in Equation 4. For more details, see [50].…”

Section: Methodsmentioning

confidence: 99%

Detection of Outliers and Imputing of Missing Values for Water Quality UV-VIS Absorbance Time Series

Plazas-Nossa

Angulo

Torres

2017

Ing.

Self Cite

View full text Add to dashboard Cite

Context: The UV-Vis absorbance collection using online optical captors for water quality detection may yield outliers and/or missing values. Therefore, pre-processing to correct these anomalies is required to improve the analysis of monitoring data. The aim of this study is to propose a method to detect outliers as well as to fill-in the gaps in time series. Method: Outliers are detected using Winsorising procedure and the application of the Discrete Fourier Transform (DFT) and the Inverse of Fast Fourier Transform (IFFT) to complete the time series. Together, these tools were used to analyse a case study comprising three sites in Colombia ( (i) Bogotá D.C. Salitre-WWTP (Waste Water Treatment Plant), influent; (ii) Bogotá D.C. Gibraltar Pumping Station (GPS); and, (iii) Itagüí, San Fernando-WWTP, influent (Medellín metropolitan area) ) analysed via UV-Vis (Ultraviolet and Visible) spectra. Results: Outlier detection with the proposed method obtained promising results when window parameter values are small and self-similar, despite that the three time series exhibited different sizes and behaviours. The DFT allowed to process different length gaps having missing values. To assess the validity of the proposed method, continuous subsets (a section) of the absorbance time series without outlier or missing values were removed from the original time series obtaining an average 12 % error rate in the three testing time series. Conclusions: The application of the DFT and the IFFT using the 10 % most important harmonics of useful values, can be advantageous for its later use in different applications, specifically for time series of water quality and quantity in urban sewer systems. One potential application would be the analysis of dry weather affecting rainy seasons, a feature achieved by detecting values that correspond to unusual behaviour in a time series. Additionally, the results hint at the potential of the method in correcting other hydrologic time series.

show abstract

“…However, the literature provides no evidence of the application of these methods for predicting UV-Vis spectrometry time series with short acquisition phases (on the order of one spectrum per minute); moreover, few cases speak on the subject from the point of view of other methods, such as the Discrete Fourier Transform (DFT) (Plazas-Nossa & Torres, 2013) or Artificial Neural Networks (ANN) (Plazas-Nossa, Avila, & Torres, 2017a).…”

Section: Introductionmentioning

confidence: 99%

Arima as a forecasting tool for water quality time series measured with UV-Vis spectrometers in a constructed wetland

Hernández

Camargo

Moreno

et al. 2017

Tecnol. cienc. agua

Self Cite

View full text Add to dashboard Cite

Arima as a tool to predict water quality using time series recorded with UV-Vis spectrometers in a constructed wetland. Water Technology and Sciences (in Spanish), 8(5), 127-139.The prediction of water quality plays a crucial role in discussions about urban drainage systems, given that the integrated management of this resource is required in order to meet human needs. The present paper uses Arima (Autoregressive Integrated Moving Average) to predict influent and effluent water quality in a constructed wetland, as well as its pollutant removal efficiency. The wetland is located on the campus of the Pontificia Universidad Javeriana in Bogotá, Colombia. Arima prediction values were based on time series obtained with UV-Vis spectrometry probes. These predictions were found to be adequate for the first 12 hours of the water quality time series for the three data sets analyzed: influent, effluent, and efficiency. Overall, none of the data had prediction errors over 15%. In separate analyses of the relative predictive errors in influent and effluent values, they were found to be less significant for UV wavelengths than for the visible range (Vis). In addition, the variability in this type of error was less for the UV range than for the Vis range, which indicates that Arima is a suitable prediction method for analyzing pollutants that fall in the UV range.

show abstract

Comparison of discrete Fourier transform (DFT) and principal component analysis/DFT as forecasting tools for absorbance time series received by UV-visible probes installed in urban sewer systems

Cited by 9 publications

References 0 publications

Interpolation in Time Series: An Introductive Overview of Existing Methods, Their Performance Criteria and Uncertainty Assessment

Interpolation in Time Series: An Introductive Overview of Existing Methods, Their Performance Criteria and Uncertainty Assessment

Detection of Outliers and Imputing of Missing Values for Water Quality UV-VIS Absorbance Time Series

Arima as a forecasting tool for water quality time series measured with UV-Vis spectrometers in a constructed wetland

Contact Info

Product

Resources

About