A Proposal of Methodology for Designing Big Data Warehouses

Abstract

Big Data warehouses are a new class of databases that largely use unstructured and volatile data for analytical purpose. Examples of this kind of data sources are those coming from the Web, such as social networks and blogs, or from sensor networks, where huge amounts of data may be available only for short intervals of time. In order to manage massive data sources, a strategy must be adopted to define multidimensional schemas in presence of fast-changing situations or even undefined business requirements. In the paper, we propose a design methodology that adopts agile and automatic approaches, in order to reduce the time necessary to integrate new data sources and to include new business requirements on the fly. The data are immediately available for analyses, since the underlying architecture is based on a virtual data warehouse that does not require the importing phase. Examples of application of the methodology are presented along the paper in order to show the validity of this approach compared to a traditional one.

Data management practices aligned with LGPD in the data science context

“…Este primeiro pilar fundamental da gestão de dados envolve a organização e o armazenamento de dados de maneira que sejam facilmente acessíveis, seguros e otimizados para performance (Di TRIA, F.;LEFONS, E.;TANGORRA, 2018).…”