Meta-Modeling of Data Sources and Ingestion Big Data Layers

Erraissi, Allae; Belangour, Abdessamad; Tragha, Abderrahim

doi:10.2139/ssrn.3185342

Cited by 13 publications

(9 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We have proposed a BD architecture enabling an efficient RS processing. This architecture is composed of six layers which are: the data sources, the ingestion layer, the Hadoop storage, the processing and management layer, and finally, the visualization layer [26]. In this paper, we focus our clarifications on the ingestion layer.…”

Section: The Data Ingestionmentioning

confidence: 99%

Big data and remote sensing: A new software of ingestion

Semlali

Amrani

2021

IJECE

View full text Add to dashboard Cite

Currently, remote sensing is widely used in environmental monitoring applications, mostly air quality mapping and climate change supervision. However, satellite sensors occur massive volumes of data in near-real-time, stored in multiple formats and are provided with high velocity and variety. Besides, the processing of satellite big data is challenging. Thus, this study aims to approve that satellite data are big data and proposes a new big data architecture for satellite data processing. The developed software is enabling an efficient remote sensing big data ingestion and preprocessing. As a result, the experiment results show that 86 percent of the unnecessary daily files are discarded with a data cleansing of 20 percent of the erroneous and inaccurate plots. The final output is integrated into the Hadoop system, especially the HDFS, HBase, and Hive, for extra calculation and processing.

show abstract

Section: The Data Ingestionmentioning

confidence: 99%

Big data and remote sensing: A new software of ingestion

Semlali

Amrani

2021

IJECE

View full text Add to dashboard Cite

show abstract

“…There are two main data sources internal and external, internal sources which are controlled by the organizations and included data about daily operations of the company that collected and stored in databases, in this case we are discussing about structured data, external sources refer to all the data that retrieved from external sources that are not controlled by the organization. (Bucur, C., 2015) (Erraissi, A., Belangour, A., & Tragha, A., 2018) Big data has different data sources, social media is the most important source, Twitter and Facebook generate very large amount of data such as tweets, profiles and likes, this data can be analyzed and provide important value, for example analysis of social media data that related to new product can provide better understanding about customer satisfaction. Log files are another source of data, for example clicks on specific website can be logged into web log files, and these logs can be analyzed to understand the online user's behavior.…”

Section: Data Sourcementioning

confidence: 99%

“…Geospatial data that generated by cell phones is another source of data that can be used by another application. 0 There are three types of data: (Erraissi, A., Belangour, A., & Tragha, A., 2018) • Structured data: it refers to the data that has fixed format and stored into rows and columns, such as data that stored into relational databases.…”

Section: Data Sourcementioning

confidence: 99%

“…(Oussous, A., Benjelloun, F. Z., Lahcen, A. A., & Belfkih, S., 2018) (Erraissi, A., Belangour, A., & Tragha, A., 2018) Big Data require to ingest, clean, process and extract an important value from the data. Different models, hardware's and technologies have been developed for Big Data to provide more trustable and accurate results, most of these technologies are open source and available to handle the volume and variety of data.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Big Data Ingestion and Preparation Tools

Alwidian¹,

Rahman²,

Gnaim³

et al. 2020

MAS

View full text Add to dashboard Cite

Developing in Big Data applications become very important in the last few years, many organizations and industries are aware that data analysis is becoming an important factor to be more competitive and discover new trends and insights. Data ingestion and preparation step is the starting point for developing any Big Data project. This paper is a review for some of the most widely used Big Data ingestion and preparation tools, it discusses the main features, advantages and usage for each tool. The purpose of this paper is to help users to select the right ingestion and preparation tool according to their needs and applications’ requirements.

show abstract

“…This triplestore uses the Jena Framework for querying RDF data. HBase is a distributed, column-oriented, scalable and fault-tolerant NoSQL [19] database where workload in terms of memory and computation (CPU) as well as storage is distributed on all machines in the HBase cluster. This NoSQL system is inspired by the BigTable's work [12], led by Google.…”

Section: Jena-hbasementioning

confidence: 99%

An Evaluation and Comparative study of massive RDF Data management approaches based on Big Data Technologies

Banane¹

2019

IJETER

View full text Add to dashboard Cite

Today, many researches are focused on the use of Big Data technologies for managing large volumes of Semantic Web data. The use of Big Data technologies such as NoSQL database management systems guarantees the scalability and high availability of Web data. The amount of data Web increases day after day and in a way excessive. To fully exploit these data and its metadata. This requires this technology of the Semantic Web to be a scalable and high performance at the level of the storage of these Web data. Many research efforts have been devoted to create and develop a distributed RDF data management system. To achieve the scalability and high performance, these systems are implemented on the basis of the Management technology of Big Data called NoSQL. In this paper, we present a comparative study of existing systems dedicated to managing large volumes of RDF data, which are generally based on NoSQL database management systems.

show abstract

Meta-Modeling of Data Sources and Ingestion Big Data Layers

Cited by 13 publications

References 4 publications

Big data and remote sensing: A new software of ingestion

Big data and remote sensing: A new software of ingestion

Big Data Ingestion and Preparation Tools

An Evaluation and Comparative study of massive RDF Data management approaches based on Big Data Technologies

Contact Info

Product

Resources

About