A Study of Heuristically-Based Parametric Performance Improvement/Optimization Algorithms for BigData Computing

Kim, Jongyeop; Park, Noh-Jin; Park, Nohpill

doi:10.1109/acit-csii-bcd.2016.087

“…The parameter space [9,11], denoted by Ω(x) is a sample space composed of parameter combinations taken from arity table and its time difference generated by map and reduce applications.…”

Section: Parameter Spacementioning

confidence: 99%

“…The e-heuristic greedy extended from the heuristically-based greedy [11] composed of several functions includes runHadoop(), run-TeraSort(); runTestDFSIO(), runMrBench(). Each function activated by the main, and in these functions, the start () method can be invoked repeatedly supported by the ProcessBuilder Class [15].…”

Section: E-heuristic Greedy Proceduresmentioning

confidence: 99%

“…Fortunately, there have been several studies conducted for efficient tuning automated manner such as Naïve exhaustive [9], random [10], and heuristically-based [11]. For the Naïve exhaustive and random approach, the TestDFSIO is used and the heuristically-based is evaluated with the TeraSort [12].…”

Section: Introductionmentioning

confidence: 99%

“…For the Naïve exhaustive and random approach, the TestDFSIO is used and the heuristically-based is evaluated with the TeraSort [12]. Since the methodology we have proposed applies single benchmarking application given by Hadoop distribution to evaluate each algorithm [11], it needs to be improved to verify more complex MapReduce task environments.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Optimized Common Parameter Set Extraction Framework by Multiple Benchmarking Applications on a Big Data Platform

Kim¹,

Kancharla²,

Seol³

et al. 2018

IJNDC

Self Cite

0

View full text Add to dashboard Cite

A B S T R A C TThis research proposes the methodology to extract common configuration parameter set by applying multiple benchmarking applications include TeraSort, TestDFSIO, and MrBench on the Hadoop distributed file system. The parameter search space conceptually conducted named Ω(x) to hold status of all parameter values and its evaluation results for every stage to eventually reduce benchmarking cost. In the process of determining parameter set for each stage, one parameter and its associated values selected which is reduced system performance in terms of overall execution time difference that are measured by multiple applications on a Hadoop cluster. The experimental results demonstrate the proposed extended greedy manner provide a feasible benchmark model for the multiple MapReduce tasks. This model classified several candidate parameter value sets that can be reduced the overall execution time by 27% of the values against Hadoop default settings. Moreover, we propose e-heuristic greedy with alternative parameter selection model to evaluate second candidate parameter value which will lead global optimum by returning back to the previous stage if local minimum is not found at the current stage compare to the previous ones.

show abstract

“…At the previous work we have developed a java application framework to modify configuration files without human intervention and applied at the several parameter evaluations [9][10][11][12].…”

Section: Hadoop Parametersmentioning

confidence: 99%

Optimized Common Parameter Set Extraction Framework by Multiple Benchmarking Applications on a Big Data Platform

Kim¹,

Kancharla²,

Seol³

et al. 2018

IJNDC

Self Cite

1

0

View full text Add to dashboard Cite

A B S T R A C TThis research proposes the methodology to extract common configuration parameter set by applying multiple benchmarking applications include TeraSort, TestDFSIO, and MrBench on the Hadoop distributed file system. The parameter search space conceptually conducted named Ω(x) to hold status of all parameter values and its evaluation results for every stage to eventually reduce benchmarking cost. In the process of determining parameter set for each stage, one parameter and its associated values selected which is reduced system performance in terms of overall execution time difference that are measured by multiple applications on a Hadoop cluster. The experimental results demonstrate the proposed extended greedy manner provide a feasible benchmark model for the multiple MapReduce tasks. This model classified several candidate parameter value sets that can be reduced the overall execution time by 27% of the values against Hadoop default settings. Moreover, we propose e-heuristic greedy with alternative parameter selection model to evaluate second candidate parameter value which will lead global optimum by returning back to the previous stage if local minimum is not found at the current stage compare to the previous ones.

show abstract

Determination of wheat kernels damaged by Fusarium head blight using monochromatic images of effective wavelengths from hyperspectral imaging coupled with an architecture self-search deep network

Yipeng

¹

,

Wenbing

²

,

Han

³

et al. 2022

View full text Add to dashboard Cite

A Study of Heuristically-Based Parametric Performance Improvement/Optimization Algorithms for BigData Computing

Cited by 3 publications

References 2 publications

Optimized Common Parameter Set Extraction Framework by Multiple Benchmarking Applications on a Big Data Platform

Optimized Common Parameter Set Extraction Framework by Multiple Benchmarking Applications on a Big Data Platform

Optimized Common Parameter Set Extraction Framework by Multiple Benchmarking Applications on a Big Data Platform

Determination of wheat kernels damaged by Fusarium head blight using monochromatic images of effective wavelengths from hyperspectral imaging coupled with an architecture self-search deep network

Contact Info

Product

Resources

About