Organizing Data Lakes for Navigation

Nargesian, Fatemeh; Pu, Ken Q.; Zhu, Erkang; Bashardoost, Bahar Ghadiri; Miller, Renée J.

doi:10.1145/3318464.3380605

Cited by 31 publications

(13 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Dataset preparation and organization KAYAK [81,82] GOODS [62,63] DS-Prox [6], DS-kNN [4] Nargesian et al [94] Juneau [130] Discover related datasets Aurum […”

Section: Maintenancementioning

confidence: 99%

Data lake concept and systems: a survey

Hai¹,

Koutras²,

Quix³

et al. 2021

Preprint

View full text Add to dashboard Cite

Although big data has been discussed for some years, it still has many research challenges, especially the variety of data. It poses a huge difficulty to efficiently integrate, access, and query the large volume of diverse data in information silos with the traditional 'schema-on-write' approaches such as data warehouses. Data lakes have been proposed as a solution to this problem. They are repositories storing raw data in its original formats and providing a common access interface. This survey reviews the development, definition, and architectures of data lakes. We provide a comprehensive overview of research questions for designing and building data lakes. We classify the existing data lake systems based on their provided functions, which makes this survey a useful technical reference for designing, implementing and applying data lakes. We hope that the thorough comparison of existing solutions and the discussion of open research challenges in this survey would motivate the future development of data lake research and practice.

show abstract

“…Dataset preparation and organization KAYAK [81,82] GOODS [62,63] DS-Prox [6], DS-kNN [4] Nargesian et al [94] Juneau [130] Discover related datasets Aurum […”

Section: Maintenancementioning

confidence: 99%

Data lake concept and systems: a survey

Hai¹,

Koutras²,

Quix³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In recent, numerous works are proposed to address open data discovery problems, including automatically discover table unionability [59] and joinability [83], [85], based on LSH and similarity measures. Nargesian and et al [58] propose a Markov approach to optimize the navigation organization as a DAG for a data lake so that the probability of finding a table by any of attributes can be maximized. In the DAG, each node of navigation DAG represents a subset of the attributes in the data lake, and an edge represents a navigation transition.…”

Section: B Data Discoverymentioning

confidence: 99%

Survive the Schema Changes: Integration of Unmanaged Data Using Deep Learning

Wang,

Zhou,

Das

et al. 2020

Preprint

View full text Add to dashboard Cite

Data is the king in the age of AI. However data integration is often a laborious task that is hard to automate. Schema change is one significant obstacle to the automation of the end-to-end data integration process. Although there exist mechanisms such as query discovery and schema modification language to handle the problem, these approaches can only work with the assumption that the schema is maintained by a database. However, we observe diversified schema changes in heterogeneous data and open data, most of which has no schema defined. In this work, we propose to use deep learning to automatically deal with schema changes through a super cell representation and automatic injection of perturbations to the training data to make the model robust to schema changes. Our experimental results demonstrate that our proposed approach is effective for two realworld data integration scenarios: coronavirus data integration, and machine log integration.

show abstract

“…Search Engines (SEs) have become some of the most necessary tools for Internet users. Generally, an SE is an information retrieval (IR) application that locates the most relevant information and then accurately conveys the results to the users based on the specified queries [1][2][3][4][5]. Meanwhile, exploratory search (ES) has a similar mechanism as SEs except that it further narrows down the search results using faceted classification.…”

Section: Introductionmentioning

confidence: 99%

“…The main research objective of this work is to determine the requirements and motivations that enhance the understanding of the FS implementation. The work reviews related works based on several perspectives, including (1) the scope of research, (2) the purpose of the study, and (3) the performance evaluation measures. Moreover, the work considered some other points of interest, including the architecture, applications, issues, research questions, motivation, recommendation criteria, and open challenges in using FS.…”

Section: Introductionmentioning

confidence: 99%

Comprehensive Review and Future Research Directions on Dynamic Faceted Search

Mahdi

Ahmad

Natiq³

et al. 2021

Applied Sciences

View full text Add to dashboard Cite

In modern society, the increasing number of web search operations on various search engines has become ubiquitous due to the significant number of results presented to the users and the incompetent result-ranking mechanism in some domains, such as medical, law, and academia. As a result, the user is overwhelmed with a large number of misranked or uncategorized search results. One of the most promising technologies to reduce the number of results and provide desirable information to the users is dynamic faceted filters. Therefore, this paper extensively reviews related research articles published in IEEE Xplore, Web of Science, and the ACM digital library. As a result, a total of 170 related research papers were considered and organized into five categories. The main contribution of this paper is to provide a detailed analysis of the faceted search’s fundamental attributes, as well as to demonstrate the motivation from the usage, concerns, challenges, and recommendations to enhance the use of the faceted approach among web search service providers.

show abstract

Organizing Data Lakes for Navigation

Cited by 31 publications

References 33 publications

Data lake concept and systems: a survey

Data lake concept and systems: a survey

Survive the Schema Changes: Integration of Unmanaged Data Using Deep Learning

Comprehensive Review and Future Research Directions on Dynamic Faceted Search

Contact Info

Product

Resources

About