Imputation of missing measurements in PV production data within constrained environments

Centeno, Iván de Paz; Garcí­a-Ordás, María Teresa; García-Olalla, Oscar; Aláiz‐Moretón, Héctor

doi:10.1016/j.eswa.2023.119510

Cited by 10 publications

(8 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is the case of the work done in [19] where the effectiveness of the transformer architecture in time series forecasting is questioned, as comparisons with linear models reveal the difficulty of the models in making accurate predictions. However, attention-based models heavily rely on positional embeddings that provide knowledge of the order and structure of the time series, and the effectiveness of these models greatly depends on the proper selection of these positional embeddings, as shown in the work of [20]. In this regard, various solutions have emerged for describing time in a time series, such as the Time2Vec method proposed in [21] that captures periodicity through learnable parameters attached to sinusoidal functions.…”

Section: A Related Workmentioning

confidence: 99%

A Granularity Invariant Method for the Classification of Energy Production Time Series in Photovoltaic Plants

Centeno¹,

García-Ordás²,

García-Olalla³

et al. 2023

IEEE Access

Self Cite

View full text Add to dashboard Cite

The treatment of photovoltaic power production time series often faces the challenge of unifying the granularity of the series when generating a predictive model. This can limit the generation of a dataset in terms of the time covered and the number of examples. In addition, models built with data of static granularities tend to show rigidity when facing granularity variations, invalidating them for scenarios different from those of the data they were trained on. To address this issue, this paper presents a novel method specifically indicated for Deep-Learning models that shows invariance to granularity called Synthesis. This operation can be added as a layer to an artificial neural network, allowing it to be applied to any power production time series and synthesizing the content of an arbitrarily long time series into a fixed-size vector which can be used for classification or regression regardless of the initial time series length. The experiments with the NIST Campus Photovoltaic dataset demonstrate the effectiveness of the method, showing an F1-Score of 1.0 for the classification of series with granularities between 2 minutes and 2 hours, and an F-Score of 1.0 for the classification of time series with variations of granularity throughout time when training with 5-minute granularity samples.

show abstract

Section: A Related Workmentioning

confidence: 99%

A Granularity Invariant Method for the Classification of Energy Production Time Series in Photovoltaic Plants

Centeno¹,

García-Ordás²,

García-Olalla³

et al. 2023

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…It efficiently learns features even in sparse datasets, making it useful for identifying data with similar patterns. Recently, there have been successful cases of MAE application in the field of missing imputation for energy data, especially when the missing rate is exceptionally high [10]. The work presented in [25] further develops the MAE model with the transformer.…”

Section: Missing Imputation Model Based On Masked Autoencoders (Maes)mentioning

confidence: 99%

“…Masked autoencoders (MAEs) can address these issues [10]. Unlike the assumption in DAE-based missing imputation, the input vector x d can be a partially observed vector.…”

Section: Masked Autoencoder (Mae)mentioning

confidence: 99%

“…To address this, recent research has proposed missing imputation methods using masked autoencoders (MAEs) [9]. Unlike traditional autoencoders, MAEs only restore observed non-missing values during the reconstruction process, leaving missing values unrecovered [10]. Instead, MAEs predict missing values based on similar patterns found in other DLP data that are similar to the given DLP data.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Denoising Masked Autoencoder-Based Missing Imputation within Constrained Environments for Electric Load Data

Jeong,

Ku,

Park

2023

Energies

View full text Add to dashboard Cite

With recent advancements in data technologies, particularly machine learning, research focusing on the enhancement of energy efficiency in residential, commercial, and industrial settings through the collection of load data, such as heat, electricity, and gas, has gained significant attention. Nevertheless, issues arising from hardware- or network-related problems can result in missing data, necessitating the development of management techniques to mitigate these challenges. Traditional methods for missing imputation face difficulties when operating in constrained environments characterized by short data collection periods and frequent consecutive missing. In this paper, we introduce the denoising masked autoencoder (DMAE) model as a solution to improve the handling of missing data, even in such restrictive settings. The proposed DMAE model capitalizes on the advantages of the denoising autoencoder (DAE), enabling effective learning of the missing imputation process, even with relatively small datasets, and the masked autoencoder (MAE), allowing for learning in environments with a high missing ratio. By integrating these strengths, the DMAE model achieves an enhanced performance in terms of missing imputation. The simulation results demonstrate that the proposed DMAE model outperforms the DAE or MAE significantly in a constrained environment where the duration of the training data is short, less than a year, and missing values occur frequently with durations ranging from 3 h to 12 h.

show abstract

“…The authors claim that their model is robust against the variability of weather factors by considering temporal and spatial coherence. De-Paz-Centeno, I. et al [18] addressed missing PV data in a constrained environment where external weather factors could not be used. They proposed an encoderdecoder structured artificial neural network (ANN)-based solution and demonstrated superior performance compared with non-parametric imputation methods in situations where predictive parametric models could not be created.…”

Section: Introductionmentioning

confidence: 99%

Weighted Average Ensemble-Based PV Forecasting in a Limited Environment with Missing Data of PV Power

Lee,

Son

2024

Sustainability

View full text Add to dashboard Cite

Photovoltaic (PV) power is subject to variability, influenced by factors such as meteorological conditions. This variability introduces uncertainties in forecasting, underscoring the necessity for enhanced forecasting models to support the large-scale integration of PV systems. Moreover, the presence of missing data during the model development process significantly impairs model performance. To address this, it is essential to impute missing data from the collected datasets before advancing with model development. Recent advances in imputation methods, including Multivariate Imputation by Chained Equations (MICEs), K-Nearest Neighbors (KNNs), and Generative Adversarial Imputation Networks (GAINs), have exhibited commendable efficacy. Nonetheless, models derived solely from a single imputation method often exhibit diminished performance under varying weather conditions. Consequently, this study introduces a weighted average ensemble model that combines multiple imputation-based models. This innovative approach adjusts the weights according to “sky status” and evaluates the performance of single-imputation models using criteria such as sky status, root mean square error (RMSE), and mean absolute error (MAE), integrating them into a comprehensive weighted ensemble model. This model demonstrates improved RMSE values, ranging from 74.805 to 74.973, which corresponds to performance enhancements of 3.293–3.799% for KNN and 3.190–4.782% for MICE, thereby affirming its effectiveness in scenarios characterized by missing data.

show abstract

Imputation of missing measurements in PV production data within constrained environments

Cited by 10 publications

References 11 publications

A Granularity Invariant Method for the Classification of Energy Production Time Series in Photovoltaic Plants

A Granularity Invariant Method for the Classification of Energy Production Time Series in Photovoltaic Plants

Denoising Masked Autoencoder-Based Missing Imputation within Constrained Environments for Electric Load Data

Weighted Average Ensemble-Based PV Forecasting in a Limited Environment with Missing Data of PV Power

Contact Info

Product

Resources

About