Outlier Data Treatment Methods Toward Smart Grid Applications

Sun, Lin; Zhou, Kaile; Yang, Shanlin

doi:10.1109/access.2018.2852759

Cited by 37 publications

(31 citation statements)

References 88 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In contrast to corrupted values, which are indicated by the low fluctuation of the net active power, outlier values were not as easy to detect. Several effective data handling techniques can be found in [33]. In the present case study, a rolling median window threshold approach is implemented, so as to conceptualize satisfactory outlier classification, while avoiding false positives (Fig.…”

Section: B Case Study Detailsmentioning

confidence: 99%

Short-Term Electric Load Forecasting With Sparse Coding Methods

et al. 2021

View full text Add to dashboard Cite

Short-term load forecasting is a key task for planning and stability of the current and future distribution grid, as it can significantly contribute to the management of energy market for ancillary services. In this paper we introduce the beneficial properties of applications of sparse representation and corresponding dictionary learning to the net load forecasting problem on a substation level. In this context, sparse representation theory can provide parsimonial predictive models, which become attractive mainly due to their ability to successfully model the input space in a self-learning manner, by interacting between theory, algorithms, and applications. Several techniques are implemented, incorporating numerous dictionary learning and sparse decomposition algorithms, and a hierarchical structured model is proposed. The concept of sparsity in each case is embedded throughout the utilization of different regularization forms which include the 0 , 1 , 2 and tree 0 norms. The observed superiority of the proposed theory, especially the one which embeds the atoms and corresponding coefficients in a tree structure, stems from the construction of the dictionary so as to represent efficiently the ambient electricity signal space and the consequent extraction of sparse basis-vectors. The performance of each model is evaluated using real hourly load measurements from a high voltage/medium voltage (HV/MV) substation and compared with that of widely used machine learning methods. The provided analytical results, verify the effectiveness of hierarchical sparse representation in short-term load forecasting applications, in terms of common accuracy indices.INDEX TERMS Generative models, hierarchical dictionaries, load forecasting, power grid, sparse representation.

show abstract

Section: B Case Study Detailsmentioning

confidence: 99%

Short-Term Electric Load Forecasting With Sparse Coding Methods

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Juan I. Guerrero et al [15] proposed an efficient system to integrate data into heterogeneous environments based on data mining techniques. While Sun, L. et al [27], proposed a method to manipulate the growing smart grid data by treating it as outlier data (a data that differs exceptionally from other observations), then categorize them into outlier rejection and outlier mining groups based on data-driven analytics and data mining techniques. Moreover, D. Kaur et al [16] proposed a tensor-based big data management scheme to reduce the data divergence problem in the dataset generated from diverse meters.…”

Section: Existing and Potential Applications In Power Consumption For Data Managementmentioning

confidence: 99%

“…One of the most famous implementations is the concept of smart homes that control all the electrical appliances to enable highly efficient consumption. It also connects these electrical appliances to various sensors from where data are collected and sent to the distributors, who can then use the data for predictive analysis [22][23][24][25][26][27][28][29][30]. Smart homes are becoming increasingly popular in developed countries because they provide a high level of automation in controlling electrical appliances while ensuring that energy is not wasted by unnecessarily turning on the appliances [31].…”

Section: Existing and Potential Applications In Power Consumption For Load Forecastingmentioning

confidence: 99%

Smart Grid Data Management in a Heterogeneous Environment with a Hybrid Load Forecasting Model

et al. 2021

View full text Add to dashboard Cite

The power consumption model can be represented in multiple dimensions, and it is proliferating to include structured and unstructured data. Dealing with such heterogeneous data and analyzing it in real-time is an ongoing challenge in the energy sector. Moreover, converting these data into useful information remains an open research area. This study focuses on modeling realistic and efficient power consumption data management in the heterogeneous environment for the Iraq energy sector and suggested a novel hybrid load forecasting model. The proposed system is named the Power Consumption Information and Analytics System (PIAS), which can perform various roles such as data acquisition from mechanical and smart meters, data federation, data management, data visualization, data analysis, and load forecasting. The proposed system has a four-tier framework (Data, Analytics, Application, and Presentation). Each layer is discussed in detail in this study to overcome the anticipated challenges. Furthermore, this study discusses the proposed system by applying two case studies. The first case study discusses power consumption data management, while the second introduces a novel hybrid load forecasting model using Fuzzy C-Means clustering, Auto Regressive Integrated Moving Average (ARIMA), and Gradient Boosted Tree Learner. The dataset used in this forecasting is based on a 1-year duration dated 1st January 2019 to 31st December 2019, on an hourly basis (365 * 24) for the Baghdad governorate. The results showed high accuracy in load forecasting with improved error rates (MAPE, MAE, and RMSE) achievements in comparison with other evaluated models such as standalone ARIMA and Gradient Boosted Trees methods.

show abstract

“…Such huge heterogeneous data combination [16][17][18][19][20] can be quite effective for the power grid when used along with the data obtained from the operational simulations using them. In general, a huge volume (terabytes) of data flows into these modern or smart grids annually [21][22][23][24][25][26][27][28][29] through various types of power network elements (e.g., sensors, smart equipment, etc.) that constantly collect the data generated by geographic information systems, SCADA systems, or weather and traffic information systems or from social networks [17][18][19][20].…”

Section: Introductionmentioning

confidence: 99%

A Novel on Transmission Line Tower Big Data Analysis Model Using Altered K-means and ADQL

Jung

2019

Sustainability

View full text Add to dashboard Cite

This study sought to propose a big data analysis and prediction model for transmission line tower outliers to assess when something is wrong with transmission line tower big data based on deep reinforcement learning. The model enables choosing automatic cluster K values based on non-labeled sensor big data. It also allows measuring the distance of action between data inside a cluster with the Q-value representing network output in the altered transmission line tower big data clustering algorithm containing transmission line tower outliers and old Deep Q Network. Specifically, this study performed principal component analysis to categorize transmission line tower data and proposed an automatic initial central point approach through standard normal distribution. It also proposed the A-Deep Q-Learning algorithm altered from the deep Q-Learning algorithm to explore policies based on the experiences of clustered data learning. It can be used to perform transmission line tower outlier data learning based on the distance of data within a cluster. The performance evaluation results show that the proposed model recorded an approximately 2.29%~4.19% higher prediction rate and around 0.8% ~ 4.3% higher accuracy rate compared to the old transmission line tower big data analysis model.

show abstract

Outlier Data Treatment Methods Toward Smart Grid Applications

Cited by 37 publications

References 88 publications

Short-Term Electric Load Forecasting With Sparse Coding Methods

Short-Term Electric Load Forecasting With Sparse Coding Methods

Smart Grid Data Management in a Heterogeneous Environment with a Hybrid Load Forecasting Model

A Novel on Transmission Line Tower Big Data Analysis Model Using Altered K-means and ADQL

Contact Info

Product

Resources

About