Detecting X-Outliers in Load Curve Data in Power Systems

Guo, Zhihui; Li, Wenyuan; Lau, Adriel; Inga-Rojas, Tito; Wang, Ke

doi:10.1109/tpwrs.2011.2167022

Cited by 28 publications

(16 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Focusing first on [S3], (8) Note that in formulating (10),F m n was eliminated using the constraintsF m n =F m n defining C F . Using (11) to eliminatē F m n [k] andF m n [k] from (5) and (6) It then follows thatF m n [k] =F n m [k] for all k ≥ 0, an identity that will be used later on. By plugging (12) in (5) The minimization (9)…”

Section: Appendix Algorithmic Constructionmentioning

confidence: 99%

“…Moreover, a major requirement for grid monitoring is robustness to outliers, i.e., data not adhering to nominal models [1], [22]. Sources of so-termed "bad data" include meter failures, as well as strikes, unscheduled generator shutdowns, and extreme weather conditions [7], [11]. Inconsistent data can also be due to malicious (cyber-) attacks that induce abrupt load changes, or counterfeit meter readings [18].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Load Curve Data Cleansing and Imputation Via Sparsity and Low Rank

Mateos

Giannakis

2013

IEEE Trans. Smart Grid

View full text Add to dashboard Cite

The smart grid vision is to build an intelligent power network with an unprecedented level of situational awareness and controllability over its services and infrastructure. This paper advocates statistical inference methods to robustify power monitoring tasks against the outlier effects owing to faulty readings and malicious attacks, as well as against missing data due to privacy concerns and communication errors. In this context, a novel load cleansing and imputation scheme is developed leveraging the low intrinsic-dimensionality of spatiotemporal load profiles and the sparse nature of "bad data." A robust estimator based on principal components pursuit (PCP) is adopted, which effects a twofold sparsity-promoting regularization through an ℓ1-norm of the outliers, and the nuclear norm of the nominal load profiles. Upon recasting the non-separable nuclear norm into a form amenable to decentralized optimization, a distributed (D-) PCP algorithm is developed to carry out the imputation and cleansing tasks using networked devices comprising the sotermed advanced metering infrastructure. If D-PCP converges and a qualification inequality is satisfied, the novel distributed estimator provably attains the performance of its centralized PCP counterpart, which has access to all networkwide data. Computer simulations and tests with real load curve data corroborate the convergence and effectiveness of the novel D-PCP algorithm.Index Terms-Advanced metering infrastructure, distributed algorithms, load curve cleansing and imputation, principal components pursuit, smart grid.

show abstract

Section: Appendix Algorithmic Constructionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Load Curve Data Cleansing and Imputation Via Sparsity and Low Rank

Mateos

Giannakis

2013

IEEE Trans. Smart Grid

View full text Add to dashboard Cite

show abstract

“…A common approach to cyberattack-resilient load forecasting is to remove malicious data through the use of anomaly detection techniques [10]. Some methods are based on descriptive analytics, which identify point, contextual, and collective anomalies such as abnormal patterns [11], [12]. Other methods are model-based and compare predicted values with observed ones [13], [14].…”

Section: Introductionmentioning

confidence: 99%

Enabling Cyberattack-Resilient Load Forecasting through Adversarial Machine Learning

Tang

Jiao

Zhang

et al. 2019

2019 IEEE Power &Amp; Energy Society General Meeting (PESGM)

View full text Add to dashboard Cite

In the face of an increasingly broad cyberattack surface, cyberattack-resilient load forecasting for electric utilities is both more necessary and more challenging than ever. In this paper, we propose an adversarial machine learning (AML) approach, which can respond to a wide range of attack behaviors without detecting outliers. It strikes a balance between enhancing a system's robustness against cyberattacks and maintaining a reasonable degree of forecasting accuracy when there is no attack. Attack models and configurations for the adversarial training were selected and evaluated to achieve the desired level of performance in a simulation study. The results validate the effectiveness and excellent performance of the proposed method.

show abstract

“…Among power and energy applications, the anomaly detection emerges as an important topic in some fields, such as electric load forecasting [11,[13][14][15], load pattern grouping [16], gas load forecasting [17] and load data cleaning [15,18,19]. Among them, some have focused on the related topics of anomaly detection for STLF.…”

Section: Introductionmentioning

confidence: 99%

“…Chakhchoukh et al proposed a robust method for outlier and break detection for seasonal ARIMA parameter estimation and forecasting the electricity consumption in France up to a day-ahead [15]. Several engineers from the British Columbia Transmission Corporation proposed several novel methods to cleanse the corrupted and missing observations in the load data [18,19]. In GEFCom2014, a winning team Jingrui Xie used a procedure based on a multiple linear regression model for outlier detection and data cleansing for STLF [14].…”

Section: Introductionmentioning

confidence: 99%

Real-time anomaly detection for very short-term load forecasting

Luo

Hong

Yue

2018

J. Mod. Power Syst. Clean Energy

View full text Add to dashboard Cite

Although the recent load information is critical to very short-term load forecasting (VSTLF), power companies often have difficulties in collecting the most recent load values accurately and timely for VSTLF applications. This paper tackles the problem of real-time anomaly detection in most recent load information used by VSTLF. This paper proposes a model-based anomaly detection method that consists of two components, a dynamic regression model and an adaptive anomaly threshold. The case study is developed using the data from ISO New England. This paper demonstrates that the proposed method significantly outperforms three other anomaly detection methods including two methods commonly used in the field and one state-of-the-art method used by a winning team of the Global Energy Forecasting Competition 2014. Finally, a general anomaly detection framework is proposed for the future research.

show abstract

Detecting X-Outliers in Load Curve Data in Power Systems

Cited by 28 publications

References 19 publications

Load Curve Data Cleansing and Imputation Via Sparsity and Low Rank

Load Curve Data Cleansing and Imputation Via Sparsity and Low Rank

Enabling Cyberattack-Resilient Load Forecasting through Adversarial Machine Learning

Real-time anomaly detection for very short-term load forecasting

Contact Info

Product

Resources

About