A New Missing Data Imputation Algorithm Applied to Electrical Data Loggers

Turrado, Concepción Crespo; Lasheras, Fernando Sánchez; Calvo-Rolle, José Luís; Piñón-Pazos, A.; Juez, Francisco Javier de Cos

doi:10.3390/s151229842

Cited by 26 publications

(12 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Against this backdrop, here, we generate randomly missing data in the weather data of the period of 2016-2017 with three missing data ratios, corresponding to the missing completely at random (MCAR) process [24]. Next, we replace the missing data with suitable values by using four different approaches, linear interpolation (LI), mode imputation (MI), k-nearest neighbors (KNN), and multivariate imputation by chain equations (MICE).…”

Section: Introductionmentioning

confidence: 99%

Analysis and Impact Evaluation of Missing Data Imputation in Day-ahead PV Generation Forecasting

Kim

2019

Applied Sciences

View full text Add to dashboard Cite

Over the past decade, PV power plants have increasingly contributed to power generation. However, PV power generation widely varies due to environmental factors; thus, the accurate forecasting of PV generation becomes essential. Meanwhile, weather data for environmental factors include many missing values; for example, when we estimated the missing values in the precipitation data of the Korea Meteorological Agency, they amounted to ~16% from 2015–2016, and further, 19% of the weather data were missing for 2017. Such missing values deteriorate the PV power generation prediction performance, and they need to be eliminated by filling in other values. Here, we explore the impact of missing data imputation methods that can be used to replace these missing values. We apply four missing data imputation methods to the training data and test data of the prediction model based on support vector regression. When the k-nearest neighbors method is applied to the test data, the prediction performance yields results closest to those for the original data with no missing values, and the prediction model’s performance is stable even when the missing data rate increases. Therefore, we conclude that the most appropriate missing data imputation for application to PV forecasting is the KNN method.

show abstract

Section: Introductionmentioning

confidence: 99%

Analysis and Impact Evaluation of Missing Data Imputation in Day-ahead PV Generation Forecasting

Kim

2019

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…The energy consumption is 190.572 KWh per day, on average. The data set detailed here was already employed by the authors in previous research [ 9 ].…”

Section: Methodsmentioning

confidence: 99%

“…The new algorithm presented in this paper hybridizes the Self-Organized Maps Neural Networks methodology with the Mahalanobis distances. The hybrid method obtained is combined with an algorithm already presented in this journal by the authors, called AAA [ 9 ], based on Multivariate Adaptive Regression Splines. The proposed methodology is new and its performance is even better than the one referenced and presented in a previous paper when applied to the same database.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

A Hybrid Algorithm for Missing Data Imputation and Its Application to Electrical Data Loggers

Turrado

Lasheras

Calvo-Rolle

et al. 2016

Sensors

Self Cite

View full text Add to dashboard Cite

The storage of data is a key process in the study of electrical power networks related to the search for harmonics and the finding of a lack of balance among phases. The presence of missing data of any of the main electrical variables (phase-to-neutral voltage, phase-to-phase voltage, current in each phase and power factor) affects any time series study in a negative way that has to be addressed. When this occurs, missing data imputation algorithms are required. These algorithms are able to substitute the data that are missing for estimated values. This research presents a new algorithm for the missing data imputation method based on Self-Organized Maps Neural Networks and Mahalanobis distances and compares it not only with a well-known technique called Multivariate Imputation by Chained Equations (MICE) but also with an algorithm previously proposed by the authors called Adaptive Assignation Algorithm (AAA). The results obtained demonstrate how the proposed method outperforms both algorithms.

show abstract

“…Unlike the studies conducted previously, we considered the approach of using a machine learning algorithm with explanatory variables. This approach can be categorized as a regression imputation [16] approach; moreover, it is possible to achieve remarkable performance according to the selected variables and the regression model used [17,18]. It must be noted that this approach is not suitable for a real-time environment owing to the requirement of additional variable collection and model training.…”

Section: Introductionmentioning

confidence: 99%

Bagging Ensemble of Multilayer Perceptrons for Missing Electricity Consumption Data Imputation

Jung

Moon

Park

et al. 2020

Sensors

View full text Add to dashboard Cite

For efficient and effective energy management, accurate energy consumption forecasting is required in energy management systems (EMSs). Recently, several artificial intelligence-based techniques have been proposed for accurate electric load forecasting; moreover, perfect energy consumption data are critical for the prediction. However, owing to diverse reasons, such as device malfunctions and signal transmission errors, missing data are frequently observed in the actual data. Previously, many imputation methods have been proposed to compensate for missing values; however, these methods have achieved limited success in imputing electric energy consumption data because the period of data missing is long and the dependency on historical data is high. In this study, we propose a novel missing-value imputation scheme for electricity consumption data. The proposed scheme uses a bagging ensemble of multilayer perceptrons (MLPs), called softmax ensemble network, wherein the ensemble weight of each MLP is determined by a softmax function. This ensemble network learns electric energy consumption data with explanatory variables and imputes missing values in this data. To evaluate the performance of our scheme, we performed diverse experiments on real electric energy consumption data and confirmed that the proposed scheme can deliver superior performance compared to other imputation methods.

show abstract

A New Missing Data Imputation Algorithm Applied to Electrical Data Loggers

Cited by 26 publications

References 28 publications

Analysis and Impact Evaluation of Missing Data Imputation in Day-ahead PV Generation Forecasting

Analysis and Impact Evaluation of Missing Data Imputation in Day-ahead PV Generation Forecasting

A Hybrid Algorithm for Missing Data Imputation and Its Application to Electrical Data Loggers

Bagging Ensemble of Multilayer Perceptrons for Missing Electricity Consumption Data Imputation

Contact Info

Product

Resources

About