DIC2FBA: Distributed Incremental Clustering with Closeness Factor Based Algorithm for Analysis of Smart Meter Data

Abstract

Due to increased civilization, smart cities, and the advent of technology, lots of buildings including commercials, residential, and other types are populating in numbers in the recent past. The electricity consumption is also affecting due to increased occupancy in these buildings. The analysis of the electricity consumption patterns will be helpful for consumers and electricity generation units to know about consumption and future requirements of electricity. As per the literature, the Incremental clustering algorithm is the best choice to handle ever-increasing data. In this research work, in the first phase, the electricity consumption data was extracted from smart meter images, and then in the second phase, the data was taken from extracted .csv files merging data from various sources together. This research proposes Distributed Incremental Clustering with Closeness Factor Based Algorithm (DIC2FBA) to update load patterns without overall daily load curve clustering. The proposed DIC2FBA has used Amazon Web Service(AWS) and Microsoft Azure HDInsight service. The AWS EC2 instance, along with the AWS S3 bucket and HdInsight, operates by clustering data from numerous sites using an iterative and incremental approach. The DIC2FBA first extracts load patterns from new data and then intergrades the existing load patterns with the new ones. Further, we have compared the findings achieved using the DIC2FBA with IK means and NFICA based on time, features, silhouette score, and the Davis Bouldin index, which indicate that our method can provide an efficient response for electricity consumption patterns analysis to end consumers via smart meters.