Implementing Materialized View of Large-Scale Power Consumption Log Using MapReduce

Ise, Yuki; Yamamoto, Sakae; Matsumoto, Shinsuke; Saiki, Sachio; Nakamura, Masahide

doi:10.1109/snpd.2013.60

Cited by 2 publications

(2 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To cope with the problem, we introduced the concept of materialized view in Scallop4SC [5]. The materialized view is a database technology that caches results of queries in an actual table to improve the response time [6].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Design and Evaluation of Materialized View as a Service for Smart City Services with Large-Scale House Log

Yamamoto

Matsumoto

Saiki

et al. 2014

IEICE Trans. Inf. & Syst.

Self Cite

View full text Add to dashboard Cite

Shintaro YAMAMOTO†a) , Nonmember, Shinsuke MATSUMOTO †b) , Sachio SAIKI †c) , and Masahide NAKAMURA †d) , Members SUMMARY Smart city services are implemented using various data collected from houses and infrastructure within a city. As the volume and variety of the smart city data becomes huge, individual services have suffered from expensive computation effort and large processing time. In order to reduce the effort and time, this paper proposes a concept of Materialized View as a Service (MVaaS). Using the MVaaS, every application can easily and dynamically construct its own materialized view, in which the raw data is converted and stored in a convenient format with appropriate granularity. Thus, once the view is constructed, the application can quickly access necessary data. In this paper, we design a framework of MVaaS specifically for large-scale house log, managed in a smart-city data platform. In the framework, each application first specifies how the raw data should be filtered, grouped and aggregated. For a given data specification, MVaaS dynamically constructs a MapReduce batch program that converts the raw data into a desired view. The batch is then executed on Hadoop, and the resultant view is stored in HBase. We present case studies using house log in a real home network system. We also conduct an experimental evaluation to compare the response time between cases with and without MVaaS. key words: large-scale, house log, materialized view, high-speed and efficient data access, MapReduce, KVS, HBase

show abstract

Section: Introductionmentioning

confidence: 99%

“…Therefore, in [5] we developed a Hadoop/MapReduce program for each application, which efficiently converts the raw data into application-specific data. The converted data is stored in an HBase table, which is used as a materialized view by the application.…”

Section: Introducing Materialized View In Scallop4scmentioning

confidence: 99%

Design and Evaluation of Materialized View as a Service for Smart City Services with Large-Scale House Log

Yamamoto

Matsumoto

Saiki

et al. 2014

IEICE Trans. Inf. & Syst.

Self Cite

View full text Add to dashboard Cite

show abstract

Parallel Maintenance of Materialized Views in Large-Scale Analytic Platforms

Sebaa

2022

International Journal of Organizational and Collective Intelligence

View full text Add to dashboard Cite

Speeding up queries processing is an important issue in database management. Materialized views are largely used to address this issue. They have been proven successful for query performance optimization. However, updating data sources of the corresponding view requires maintaining the related views. Therefore, a view maintenance strategy is required. This paper presents a novel approach for materialized view maintenance that overcomes the limitations of prior approaches using a parallel “Divide and Conquer” strategy. We modeled the view maintenance problem using a new concept called “Multiple-Views-Matrix” as a matrix that brings up all affected views and their corresponding base relations. Moreover, we introduce a new algorithm for performing maintenance of multiple views, the proposed algorithm is able to use multiple parallelism and recursivity; this allows it to maintain multiple views and to process several updates at the same time. We show that our method provides a significant improvement in terms of maintenance process cost.

show abstract

Implementing Materialized View of Large-Scale Power Consumption Log Using MapReduce

Cited by 2 publications

References 7 publications

Design and Evaluation of Materialized View as a Service for Smart City Services with Large-Scale House Log

Design and Evaluation of Materialized View as a Service for Smart City Services with Large-Scale House Log

Parallel Maintenance of Materialized Views in Large-Scale Analytic Platforms

Contact Info

Product

Resources

About