Integrating Online Data for Smart City Data Marts

Abstract

Abstract. The development of smart city infrastructures is seen as an important strategic component for many countries with applications in the areas of government services, healthcare, transport and traffic management, energy, water, and the management of waste services. In general, each of these infrastructural components generates data as it delivers its service. The usage of this data is often crucial both to the continued development of the service and for forward planning. While the data sources are often not complex in structure, traditional decision support systems are not built to handle continuous data streams, new or disappearing data sources, or the integration of multiple online data sources. In this work, we develop a flexible ETL process to quickly process and integrate online smart city data sources to deliver information that can be used in close to real-time.

Representative Sample Extraction from Web Data Streams

“…These processing tasks include acquisition and interpretation, transformation, integration, and analysis or machine learning of data from sensors, devices, vehicles etc. There have been a number of approaches to building smart city applications [5] and to cluster or integrate query graphs on smart city data [10,12]. Decision makers who use smart city applications require OLAP type services to generate the datasets from a warehouse upon which they make their decision.…”