The Large Scale Data Facility: Data Intensive Computing for Scientific Experiments

2014 IEEE International Energy Conference (ENERGYCON)

Bach

et al. 2014

As the power system management currently advances from rigid to flexible, highly sophisticated monitoring and control is required in the future. By developing the Electrical Data Recorder (EDR) and preparing the device for comparative tests in low voltage power network at multiple locations we intend to contribute to a more detailed analysis and to provide a fundamental database for simulation and investigation. Using EDR devices, high rate captures of low voltage time series are acquired synchronously and are stored in a central large database without any information reduction. We performed extensive tests according to the IEC 61000-4-30 standard in order to assure the comparability of the measured values. Special attention was paid to the synchronization for achieving temporal coincidence of values measured at different locations. We developed methods for large scale data storage and interfaces for 3 rd party simulation packages. In this paper we present the preparation of the device and the test procedures as well as the results. We report on the methods we already provide for data management and analysis. We conclude that the EDR provides sufficient accuracy and precision for comparative tests; our data storage and retrieval methods are fast and reliable; and we did not find 3 rd party simulation tools appropriate for our needs, yet. However, the measurement system is ready for productive use in the low voltage supply grid.

Section: A Large Scale Data Facility and Hadoopmentioning

confidence: 99%

Preparing the electrical data recorder for comparative power network measurements

2014 IEEE International Energy Conference (ENERGYCON)

Bach

et al. 2014

“…Concerning EDR, data are stored in the Large Scale Data Facility (LSDF) at KIT [29] which contains a Hadoop File System (HDFS) for big data analyses. General Parallel File System (GPFS) and a Scale Out Network Attached Storage (SONAS) for band-secured long-term archiving.…”

Section: Gds Storage and Retrievalmentioning

confidence: 99%

Data processing of high-rate low-voltage distribution grid recordings for smart grid monitoring and analysis

EURASIP J. Adv. Signal Process.

Bach

et al. 2015

Power networks will change from a rigid hierarchic architecture to dynamic interconnected smart grids. In traditional power grids, the frequency is the controlled quantity to maintain supply and load power balance. Thereby, high rotating mass inertia ensures for stability. In the future, system stability will have to rely more on real-time measurements and sophisticated control, especially when integrating fluctuating renewable power sources or high-load consumers like electrical vehicles to the low-voltage distribution grid. In the present contribution, we describe a data processing network for the in-house developed low-voltage, high-rate measurement devices called electrical data recorder (EDR). These capture units are capable of sending the full high-rate acquisition data for permanent storage in a large-scale database. The EDR network is specifically designed to serve for reliable and secured transport of large data, live performance monitoring, and deep data mining. We integrate dedicated different interfaces for statistical evaluation, big data queries, comparative analysis, and data integrity tests in order to provide a wide range of useful post-processing methods for smart grid analysis. We implemented the developed EDR network architecture for high-rate measurement data processing and management at different locations in the power grid of our Institute. The system runs stable and successfully collects data since several years. The results of the implemented evaluation functionalities show the feasibility of the implemented methods for signal processing, in view of enhanced smart grid operation.

“…Currently, the EDR time series are stored in the Hadoop cluster of KIT's Large Scale Data Facility (LSDF) [19]. The installed Hadoop file system (HDFS) currently provides a capacity of 100 TB.…”

Section: B Data Flowmentioning

confidence: 99%

Introducing the Electrical Data Recorder as a new capturing device for power grid analysis

2012 IEEE International Workshop on Applied Measurements for Power Systems (AMPS) Proceedings

Suess

et al. 2012

Power systems are facing fundamental changes concerning functionality and dynamics today. However, detailed data of the system are property of the suppliers or distributors and are not easily available to researchers at present. In order to understand the system dependencies we propose to derive state information by combining high-rate low-voltage time series captures at different locations together with a simulation model of the grid. We take the island network like KIT campus as our starting investigation site. In a first step we developed the Electrical Data Recorder (EDR), which is capable of recording three phase voltage time series at up to 25 kHz synchronously. All data are stored in a large scale database facility (LSDF) for subsequent usage. We intend to derive a simulation model from the comparison of the voltage characteristics to periodic smart meter measurements as the indication of the load status. In this article we introduce the new recording device and present first results.Index Terms-power system analysis, smart grid modeling, power network simulation, large scale data 1 Manuscript received August 3, 2012.Heiko Maass is with the Karlsruhe