Design and Evaluation of Materialized View as a Service for Smart City Services with Large-Scale House Log

Abstract

Shintaro YAMAMOTO†a) , Nonmember, Shinsuke MATSUMOTO †b) , Sachio SAIKI †c) , and Masahide NAKAMURA †d) , Members SUMMARY Smart city services are implemented using various data collected from houses and infrastructure within a city. As the volume and variety of the smart city data becomes huge, individual services have suffered from expensive computation effort and large processing time. In order to reduce the effort and time, this paper proposes a concept of Materialized View as a Service (MVaaS). Using the MVaaS, every application can easily and dynamically construct its own materialized view, in which the raw data is converted and stored in a convenient format with appropriate granularity. Thus, once the view is constructed, the application can quickly access necessary data. In this paper, we design a framework of MVaaS specifically for large-scale house log, managed in a smart-city data platform. In the framework, each application first specifies how the raw data should be filtered, grouped and aggregated. For a given data specification, MVaaS dynamically constructs a MapReduce batch program that converts the raw data into a desired view. The batch is then executed on Hadoop, and the resultant view is stored in HBase. We present case studies using house log in a real home network system. We also conduct an experimental evaluation to compare the response time between cases with and without MVaaS. key words: large-scale, house log, materialized view, high-speed and efficient data access, MapReduce, KVS, HBase