Social Media Data Integration: From Data Lake to NoSQL Data Warehouse

Dabbèchi, Hichem; Haddar, Nader; Elghazel, Haytham; Haddar, Kais

doi:10.1007/978-3-030-71187-0_64

Cited by 6 publications

(4 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A. Arman [29] dealt with transferring a NoSQL database to a different NoSQL database that has distinct features. The method outlined in [8] enables the incorporation of NoSQL databases into a NoSQL data warehouse with a star-shaped structure, where connections are restricted to relationships between dimensions and facts. Upon analyzing the selected research papers, certain gaps in the research are identified.…”

Section: Related Work D L Moody Et Almentioning

confidence: 99%

NODW Framework for Data Warehousing -A NoSQL Big Data Perspective

Imran,

Mahmood,

Khan

et al. 2024

Preprint

View full text Add to dashboard Cite

The idea of data warehousing is data model independence and does not require the adoption of any data model. Nevertheless, relational databases are assumed to be used in the common notion of data warehouses. Typically, it is thought that data warehouses are relational data storage, and relational database management systems (RDBMSs) are used to process them. Data warehousing frequently features Structured Query Language (SQL) reporting capabilities, enabling access to the data in a standard manner. "Not only SQL" (NoSQL) databases are viewed as high-speed data structures appropriate for filtering and lookup operations suitable for complex processing tasks. Traditional database systems are incapable of dealing with vast volumes of data for knowledge discovery and complex analytics. This incapability is facing a paradigm shift in technologies, techniques, concepts, and methods. The key problem is to achieve a good balance between the characteristics of classical data warehouses employing relational database management systems and the potential afforded by NoSQL database management systems in a big data environment. This paper covers the integration of disparate big data technologies using an opensource NoSQL columnar database management system to address the possibility of constructing data warehouse (DW) solutions in a big data environment.

show abstract

Section: Related Work D L Moody Et Almentioning

confidence: 99%

NODW Framework for Data Warehousing -A NoSQL Big Data Perspective

Imran,

Mahmood,

Khan

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…However, big data lake, recent big news, depicts a recommended solution for dealing with heterogeneous datasets in any format, structured, semi-structured, or unstructured. Thus, numerous contributions, such as No-SQL databases, have been offered for the optimization of processing times on the Big Data Lake [1] [2] [3]. Faced with this challenge, this paper aims to investigate the data lake optimization queries through an efficient and powerful approach based on the Grover algorithm.…”

Section: Introductionmentioning

confidence: 99%

Grover’s Algorithm for Data Lake Optimization Queries

Cherradi¹,

Haddadi²

2022

IJACSA

View full text Add to dashboard Cite

Now-a-days, the use of No-SQL databases is one of the potential options for storing and processing big data lakes. However, searching for large data in No-SQL databases is a complex and time-consuming task. Further, information retrieval from big data management suffers in terms of execution time. To reduce the execution time during the search process, we propose a fast and suitable approach based on the quantum Grover algorithm, which represents one of the best-known approaches for searching in an unstructured database and resolves the unsorted search query in O (√ ) time complexity. To assess our proposal, a comparative study with linear and binary search algorithms was conducted to prove the effectiveness of Grover's algorithms. Then, we perform extensive experiment evaluations based on ibm_qasm_simulator for searching one item out of eight using Grover's search algorithm based on three qubits. The experiments outcomes revealed encouraging results, with an accuracy of 0.948, well in accordance with the theoretical result. Moreover, a discussion of the sensitivity of Grover's algorithm through different iterations was carried out. Then, exceeding the optimal number of iterations round ( √ ), induces low accuracy of the marked state. Furthermore, the incorrect selection of this parameter can outline the solution.

show abstract

“…Data integration is a challenging task, even more nowadays where we deal with the V's for big data, such as variety, variability, and volume (Searls [1]; Lin et al [2]; Alserafi et al [3]). Regarding the variety of data to be integrated into data lakes, having different types of data can be considered one of the most difficult challenges, even more because most datasets may contain unstructured or semi-structured information (Dabbèchi et al [4]). According to Hai, Quix, and Zhou [5], it is very onerous to perform interesting integrative queries over distinct types of datasets.…”

Section: Introductionmentioning

confidence: 99%

Untitled

2022

IJDKP

View full text Add to dashboard Cite

When we work in a data lake, data integration is not easy, mainly because the data is usually stored in raw format. Manually performing data integration is a time-consuming task that requires the supervision of a specialist, which can make mistakes or not be able to see the optimal point for data integration among two or more datasets. This paper presents a model to perform heterogeneous in-memory data integration in a Hadoop-based data lake based on a top-k set similarity approach. Our main contribution is the process of ingesting, storing, processing, integrating, and visualizing the data integration points. The algorithm for data integration is based on the Overlap coefficient since it presented better results when compared with the set similarity metrics Jaccard, Sørensen-Dice, and the Tversky index. We tested our model applying it on eight bioinformatics-domain datasets. Our model presents better results when compared to an analysis of a specialist, and we expect our model can be reused for other domains of datasets.

show abstract

Social Media Data Integration: From Data Lake to NoSQL Data Warehouse

Cited by 6 publications

References 15 publications

NODW Framework for Data Warehousing -A NoSQL Big Data Perspective

NODW Framework for Data Warehousing -A NoSQL Big Data Perspective

Grover’s Algorithm for Data Lake Optimization Queries

Untitled

Contact Info

Product

Resources

About