V for variety: Lessons learned from complex smart cities data harmonization and integration

Abstract

Abstract-With emerging trends for Internet of Things (IoT) and Smart Cities, complex data transformation, aggregation and visualization problems are becoming increasingly common. These tasks support improved business intelligence, analytics and enduser access to data. However, in most cases developers of these tasks are presented with challenging problems including noisy data, diverse data formats, data modeling and increasing demand for sophisticated visualization support. This paper describes our experiences with just such problems in the context of Household Travel Surveys data integration and harmonization. We describe a common approach for addressing these harmonizations. We then discuss a set of lessons that we have learned from our experience that we hope will be useful for others embarking on similar problems. We also identify several key directions and needs for future research and practical support in this area.

Smart City Data in Urban Wellbeing Estimation

“…The demand for adopting data management frameworks for smart city data is growing as data is diversifying, increasing in size and becoming difficult to manage [2]. Recent work has emphasized the creation of frameworks that include data collection, processing, analysis, management, and visualization features for robust smart city applications [3].…”

Data Service API Design for Data Analytics

“…However, there is a semantic gap for clients to be navigated by hyperlink in different domain context. To remedy this design flaw in HATEOAS implementation, some hypermedia specifications such as AtomPub 5 and OpenSearch 6 are tailored to achieve the specific application goals. However, the semantics of these domainspecific media types are implicit and generic [60].…”