Big data storage technologies: a survey

Siddiqa, Aisha; Karim, Ahmad; Gani, Abdullah

doi:10.1631/fitee.1500441

Cited by 93 publications

(41 citation statements)

References 67 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Another problem related to storing data on external platforms is the volume that they occupy. It is especially important in the case of huge data volumes, which we deal with within the context of Big Data [16]. The smaller the data set is, the faster it will be processed, and the cost of the storage service provided is lower.…”

Section: Compressionmentioning

confidence: 99%

See 1 more Smart Citation

A New Way to Store Simple Text Files

Lawnik¹,

Pełka²,

Kapczyński³

2020

Algorithms

View full text Add to dashboard Cite

In the era of ubiquitous digitization, the Internet of Things (IoT), information plays a vital role. All types of data are collected, and some of this data are stored as text files. An important aspect—regardless of the type of data—is related to file storage, especially the amount of disk space that is required. The less space is used on storing data sets, the lower is the cost of this service. Another important aspect of storing data warehouses in the form of files is the cost of data transmission needed for file transfer and its processing. Moreover, the data that are stored should be minimally protected against access and reading by other entities. The aspects mentioned above are particularly important for large data sets like Big Data. Considering the above criteria, i.e., minimizing storage space, data transfer, ensuring minimum security, the main goal of the article was to show the new way of storing text files. This article presents a method that converts data from text files like txt, json, html, py to images (image files) in png format. Taking into account such criteria as the output size of the file, the results obtained for the test files confirm that presented method enables to reduce the need for disk space, as well as to hide data in an image file. The described method can be used for texts saved in extended ASCII and UTF-8 coding.

show abstract

Section: Compressionmentioning

confidence: 99%

“…Big Data storage systems can be characterized by specifying both strengths and weaknesses, which were presented in [16]. Among the advantages, there is the support of heterogeneous structured data, simultaneous accessibility, and high fault tolerance.…”

mentioning

confidence: 99%

A New Way to Store Simple Text Files

Lawnik¹,

Pełka²,

Kapczyński³

2020

Algorithms

View full text Add to dashboard Cite

show abstract

“…All these projects similarly aim to semantically integrate heterogeneous data coming from big data sources such as sentinel data, including data provided by citizens through social media. However, data storage and management with traditional data management platforms is difficult [25]. As the number of data sources and the type of data stores augment, data access needs to be made easier for better real-time prediction…”

Section: Introductionmentioning

confidence: 99%

PREDICAT: A Semantic Service-Oriented Platform for Data Interoperability and Linking in Earth Observation and Disaster Prediction

Masmoudi

Taktak

Lamine³

et al. 2018

2018 IEEE 11th Conference on Service-Oriented Computing and Applications (SOCA)

View full text Add to dashboard Cite

The increasing volume of data generated by earth observation programs such as Copernicus, NOAA, and NASA Earth Data, is overwhelming. Although these programs are very costly, data usage remains limited due to lack of interoperability and data linking. In fact, multi-source and heterogeneous data exploitation could be significantly improved in different domains especially in the natural disaster prediction one. To deal with this issue, we introduce the PREDICAT project that aims at providing a semantic service-oriented platform to PREDIct natural CATastrophes. The PREDICAT platform considers (1) data access based on web service technology; (2) ontology-based interoperability for the environmental monitoring domain; (3) data integration and linking via big data techniques; (4) a prediction approach based on semantic machine learning mechanisms. The focus in this paper is to provide an overview of the PREDICAT platform architecture. A scenario explaining the operation of the platform is presented based on data provided by our collaborators, including the international intergovernmental Sahara and Sahel Observatory (OSS).

show abstract

“…Big Data refers to collection of huge data sets with a great diversity in types so that it becomes difficult to process by state-of-the-art data processing approaches or platforms [21]. More generally, we can say that it is formidable to perform capture, preparation, analysis and visualization on big data by current technologies.…”

Section: Introductionmentioning

confidence: 99%

On the analysis of big data indexing execution strategies

Siddiqa

Karim

Saba

et al. 2017

IFS

Self Cite

View full text Add to dashboard Cite

Abstract. Efficient response to search queries is very crucial for data analysts to obtain timely results from big data spanned over heterogeneous machines. Currently, a number of big-data processing frameworks are available in which search operations are performed in distributed and parallel manner. However, implementation of indexing mechanism results in noticeable reduction of overall query processing time. There is an urge to assess the feasibility and impact of indexing towards query execution performance. This paper investigates the performance of state-of-the-art clustered indexing approaches over Hadoop framework which is de facto standard for big data processing. Moreover, this study leverages a comparative analysis of nonclustered indexing overhead in terms of time and space taken by indexing process for varying volume data sets with increasing Index Hit Ratio. Furthermore, the experiments evaluate performance of search operations in terms of data access and retrieval time for queries that use indexes. We then validated the obtained results using Petri net mathematical modeling. We used multiple data sets in our experiments to manifest the impact of growing volume of data on indexing and data search and retrieval performance. The results and highlighted challenges favorably lead researchers towards improved implication of indexing mechanism in perspective of data retrieval from big data. Additionally, this study advocates selection of a non-clustered indexing solution so that optimized search performance over big data is obtained.

show abstract

Big data storage technologies: a survey

Cited by 93 publications

References 67 publications

A New Way to Store Simple Text Files

A New Way to Store Simple Text Files

PREDICAT: A Semantic Service-Oriented Platform for Data Interoperability and Linking in Earth Observation and Disaster Prediction

On the analysis of big data indexing execution strategies

Contact Info

Product

Resources

About