A Comparison of ORC-Compress Performance with Big Data Workload on Virtualization

Rattanaopas, Kritwara; Kaewkeerat, Sureerat; Chuchuen, Yanapat

doi:10.4028/www.scientific.net/amm.855.153

Cited by 5 publications

(4 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When it comes to multi-dimensional complex associations, the efficiency is very low. To address this issue, a Solr-based Hbase massive data secondary index scheme is proposed in [16]. Besides, a secondary index is established for Hbase [8], such that the query efficiency can be improved.…”

Section: Related Workmentioning

confidence: 99%

Towards Query Performance Improvement in Big Data Environment

Li¹,

Luo²,

Song³

et al. 2022

Proceedings of the 2022 3rd International Conference on Big Data and Informatization Education (ICBDIE 2022)

View full text Add to dashboard Cite

In recent years, big data has exploded in the communications industry, especially in the 4G industry card data. However, the large amount of data and processing dimensions of 4G industry card data incur a large overhead of query performance. In order to improve its query efficiency, this paper studies the impact of data organization on query performance, and proposes a data organization framework suitable for 4G industry card data. The proposed framework integrates data organization patterns of Spark SQL and HBase + Phoenix, and can automatically select the appropriate data organization for different business types. The experimental results show that the proposed data organization framework improves the query efficiency by 35.35% on average.

show abstract

Section: Related Workmentioning

confidence: 99%

Towards Query Performance Improvement in Big Data Environment

Li¹,

Luo²,

Song³

et al. 2022

Proceedings of the 2022 3rd International Conference on Big Data and Informatization Education (ICBDIE 2022)

View full text Add to dashboard Cite

show abstract

“…This has saved a lot of manpower and time costs for the enterprise. (b) The original data types are diverse and have different formats, such as CSV, TXT, JSON, Parquet, ORC, and other formats. , In addition, data sources are also diverse, including relational MySQL, Kafka, Hbase, and other data sources. DLI is fully compatible with the Spark ecosystem, naturally supporting multiple data sources and formats.…”

Section: Introductionmentioning

confidence: 99%

Construction and Application of a Big Data System for Regional Lakes in Coalbed Methane Development

Wang¹,

Adenutsi

Wang³

et al. 2023

ACS Omega

View full text Add to dashboard Cite

With the rapid development and widespread application of big data and artificial intelligence, the upgrading of digital and intelligent industries has been rapidly popularized in the oil and gas industry. First, based on the theory of ″regional data lake″, the digital nature of the CBM governance system is analyzed, and the optimization model of CBM governance for different data types is established. Second, considering the geological characteristics and development mode of the CBM reservoir, the regional data lake expansion model is established. Third, a theoretical model of coupling ″on-site data, laboratory data, management data, and data management system″ has been established. The research shows the following: (a) The CBM governance system based on the regional data lake can be divided into four parts: basic support, data life cycle, core governance areas, and governance strategy support. (b) The coupling of the coalbed methane governance model with the BP neural network model in this article has good application results. (c) The computational efficiency of this model has been improved by 12%, which has broad application prospects.

show abstract

“…13 Facing the new unconventional gas reservoir type of deep coalbed methane, there is an urgent need to solve numerous basic theories and technical bottlenecks. 14 This is the only way to achieve gas storage and production increase and effective economic development. 15 periphery, empowering cooperation, expanding new energy (sources), and digital management."…”

Section: Introductionmentioning

confidence: 99%

“…The fracturing fluid system still cannot meet the requirements of low damage and fouling transformation, and further technology development is needed . Facing the new unconventional gas reservoir type of deep coalbed methane, there is an urgent need to solve numerous basic theories and technical bottlenecks . This is the only way to achieve gas storage and production increase and effective economic development .…”

Section: Introductionmentioning

confidence: 99%

Intelligent Decision-Making System for Integrated Geological and Engineering of Deep Coalbed Methane Development

Wang,

Huang

et al. 2023

Energy Fuels

View full text Add to dashboard Cite

Coalbed methane (CBM) exploration and development is becoming increasingly complex, and the deterioration of resource quality is intensifying. In this paper, first, based on the research results of data regional lake and fine description of gas reservoirs, artificial intelligence technology was applied to deep mining and analysis of comprehensive data of CBM reservoirs. Second, an innovative integrated CBM geology engineering intelligent decision-making system based on big data analysis algorithms was developed. Third, the fast single-well production prediction and master controlling factor analysis driven by big data is achieved. Fourth, comprehensive sweet spot analysis was conducted on CBM reservoirs by combining geological and engineering factors. The fracturing parameters based on the analysis of the main control factors of production after fracturing of CBM wells were optimized. The system is promoted and applied in Daning block on the east edge of the Ordos Basin. Results show that (a) the system has significantly improved the efficiency of gas reservoir research and the effectiveness of engineering decisions, and (b) it provides a powerful new method and system support for fully understanding the resource potential of gas reservoirs and comprehensively promoting the deep application of intelligence in scientific research and production.

show abstract

A Comparison of ORC-Compress Performance with Big Data Workload on Virtualization

Cited by 5 publications

References 5 publications

Towards Query Performance Improvement in Big Data Environment

Towards Query Performance Improvement in Big Data Environment

Construction and Application of a Big Data System for Regional Lakes in Coalbed Methane Development

Intelligent Decision-Making System for Integrated Geological and Engineering of Deep Coalbed Methane Development

Contact Info

Product

Resources

About