Measuring the Performance of Data Placement Structures for  MapReduce-based Data Warehousing Systems

Makki, S. Kami; Hasan, Mohammad Rakibul

doi:10.17781/p002371

Cited by 1 publication

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…RCFile [27] is the most primitive column-based data file format and ORCFile is an optimized version of the same. Although, ORCFile [13] is considered apt for applications with ACID transactions and offers fast access with its indexing feature, Parquet is promoted by Cloudera [36] and is known to perform best with Spark [8].…”

Section: Datamentioning

confidence: 99%

File Format s for Big Data Storage Systems

Khan*,

Alam*

2019

IJEAT

View full text Add to dashboard Cite

Big data is one of the most influential technologies of the modern era. However, in order to support maturity of big data systems, development and sustenance of heterogeneous environments is requires. This, in turn, requires integration of technologies as well as concepts. Computing and storage are the two core components of any big data system. With that said, big data storage needs to communicate with the execution engine and other processing and visualization technologies to create a comprehensive solution. This brings the facet of big data file formats into picture. This paper classifies available big data file formats into five categories namely text-based, row-based, column-based, in-memory and data storage services. It also compares the advantages, shortcomings and possible use cases of available big data file formats for Hadoop, which is the foundation for most big data computing technologies. Lastly, it provides a discussion on tradeoffs that must be considered while choosing a file format for a big data system, providing a framework for creation for file format selection criteria.

show abstract

Section: Datamentioning

confidence: 99%