Machine Learning in Data Lake for Combining Data Silos

Wibowo, Merlinda; Sulaiman, Sarina; Shamsuddin, Siti Mariyam

doi:10.1007/978-3-319-61845-6_30

Cited by 16 publications

(12 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The first author to reference Dixon's Concept in academic context was O'Leary [63], in a paper published in 2014. We also discovered the most cited academic definition for data lakes is from Terrizzano et al [87], mentioned in twelve papers [5], [26], [27], [33], [34], [52], [66], [78], [82], [84], [91], [93]. System architectures are formed by a set of tools that work together to achieve the environment objective.…”

Section: Resultsmentioning

confidence: 99%

“…Initial Accepted Scopus 108 53 papers: [1]- [3], [5], [9], [10], [13]- [19], [23]- [29], [31]- [33], [37], [40], [45], [49], [50], [57], [60]- [66], [68], [70], [71], [73], [76]- [78], [81]- [84], [88], [90], [91], [93]- [95] Springer 222 20 papers: [4], [6], [12], [21], [30], [36], [38], [39], [41]- [43], [47], [51], [53], [69], [74], [79], [85], [86], [92] Google Scholar 197 6 papers:...…”

Section: Sourcementioning

confidence: 99%

See 1 more Smart Citation

A Mapping Study about Data Lakes: An Improved Definition and Possible Architectures

Couto¹,

Borges²,

Ruiz³

et al. 2019

International Conferences on Software Engineering and Knowledge Engineering

View full text Add to dashboard Cite

In the past few years, data lakes emerged as a trending topic in big data technologies. Although literature presents different points of view related to its functionalities, it serves mainly to store a variety of data in a big data context. In this paper, we aim to identify and analyze data lake definitions and possible architectures. Our methodology was composed of a systematic literature mapping based on PRISMA, software engineering best practices to perform reviews, and Kappa method to assess results' quality. We performed the search in eight different electronic databases to achieve a wide variety of publishers in Computer Science. We first identified 662 papers matching our search criteria; after filtering, we selected 87 papers for review. We found that the term data lakes was first defined by James Dixon in 2010. We also found that the term is often related to raw data repositories. From the identified definitions, we propose a new one as a means to better state what data lakes refer to and improve how the community use them. Moreover, we foind that Hadoop and its ecosystem compose the most used toolset to create data lakes, revealing that this is the mainstream in architectures for data lakes as of today's available technologies.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Sourcementioning

confidence: 99%

A Mapping Study about Data Lakes: An Improved Definition and Possible Architectures

Couto¹,

Borges²,

Ruiz³

et al. 2019

International Conferences on Software Engineering and Knowledge Engineering

View full text Add to dashboard Cite

show abstract

“…present Rough Set as a predicting method, because the significant features can be well considered by using reduce computation and non-critical features in silos will be eliminated. [15]. The combination of different data sources provides the need for metadata management.…”

Section: Approaches and Techniques In The Big Data Contextmentioning

confidence: 99%

Extracting knowledge patterns in a data lake for management effectiveness

Cheng

Wang

2020

E3S Web Conf.

View full text Add to dashboard Cite

With the correlation collision between different types of data becomes more and more intense, a meaningful and far-reaching data revolution has arrived. Enterprises urgently require a hybrid data platform that can effectively break data silos, and unify data aggregation and sharing. Once the data lake was born, it has been a promising method for enterprises to profoundly improve their Business Intelligence. In this paper, we combine principle component analysis (PCA) with a network-based approach to extract a visual knowledge pattern from data sources in data lake, so as to improve management effectiveness.

show abstract

“…Several techniques, for instance, machine learning and predictive analysis, among others, can be applied to this data. Moreover, a data lake is highly scalable, very accessible, and quick to update; it does not require planning or prior knowledge about the data, since it assumes that analysis will happen later, on-demand [15].…”

Section: Introductionmentioning

confidence: 99%