Learning-Based Adaptive Imputation Methodwith kNN Algorithm for Missing Power Data

Abstract

This paper proposes a learning-based adaptive imputation method (LAI) for imputing missing power data in an energy system. This method estimates the missing power data by using the pattern that appears in the collected data. Here, in order to capture the patterns from past power data, we newly model a feature vector by using past data and its variations. The proposed LAI then learns the optimal length of the feature vector and the optimal historical length, which are significant hyper parameters of the proposed method, by utilizing intentional missing data. Based on a weighted distance between feature vectors representing a missing situation and past situation, missing power data are estimated by referring to the k most similar past situations in the optimal historical length. We further extend the proposed LAI to alleviate the effect of unexpected variation in power data and refer to this new approach as the extended LAI method (eLAI). The eLAI selects a method between linear interpolation (LI) and the proposed LAI to improve accuracy under unexpected variations. Finally, from a simulation under various energy consumption profiles, we verify that the proposed eLAI achieves about a 74% reduction of the average imputation error in an energy system, compared to the existing imputation methods.

Bagging Ensemble of Multilayer Perceptrons for Missing Electricity Consumption Data Imputation

“…For the accuracy comparison, we used the mean absolute percentage error (MAPE) and root mean square error (RMSE). MAPE and RMSE can be defined using Equations (8) and (9), respectively. Here, A t and F t are the actual and prediction values, respectively, and n is the number of data used in a test.…”

“…However, guaranteeing accuracy in the energy consumption data is not trivial because there are several factors that result in missing data [6,7]. For example, malfunctions of the device and signal transmission errors are typical sources of missing data [8]. This missing value problem decreases the prediction accuracy and results in inferior performance for the forecasting methods that are based on consecutive values, such as the autoregressive integrated moving average [9,10].…”

“…This method avoided the drawbacks of both the LI and the HA; therefore, it delivered better accuracy than those of the two individual methods. Kim et al [8] introduced a learning-based adaptive imputation (LAI) method based on feature vectors of the data points surrounding the missing data point. By utilizing intentionally generated missing data, the LAI method learns the optimal length of the feature vector and the range of the historical data that must be considered during the imputation.…”

Artificial Intelligence Techniques in Smart Grid: A Survey

“…Support vector machine (SVM) is one of the most robust classification models proposed by Vapnik [28]. The k-nearest neighbors (KNN) algorithm, which is very fast for training, is also used for classification and regression in smart grid systems [29][30][31]. The decision tree learning model and logistic regression, which are very easy to interpret and implement, have also been widely adapted in smart gird systems [32,33].…”

Denoising Autoencoder-Based Missing Value Imputation for Smart Meters

“…To overcome the disadvantages of LI and HA methods, optimally weighted average imputation method was proposed in [14], where missing values are imputed by the weighted sum of both LI and HA results. In [15], the authors developed a learning-based adaptive missing data imputation method (LAI) by applying kNN algorithm to the optimal length of historical data. Then, the extended LAI method, which combines LAI and LI, was proposed to alleviate the unexpected variations in the missing durations.…”