Intelligent prediction of execution times

Tetzlaff, Dirk; Glesner, Sabine

doi:10.1109/icoia.2013.6650262

Cited by 8 publications

(15 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The RAM size can be measured one time per platform, but the estimation of RSS at compile time is not straightforward. For accurate measurement of RSS, an application is required to be profiled each time before passing through the detection framework, which will affect the major objective of requirement identification before the actual execution [28]- [31]. On several occasions, one-time profiling is justified for optimizing the same application repeatedly [28].…”

Section: A Volume Detectionmentioning

confidence: 99%

“…, [33], which may be less accurate at the benefit of minimal overhead. The training set builds the regression model offline, which is used by the incoming applications for online RSS predictions [28], [31], [33]. The RSS is not the same for different applications involving the same data size.…”

Section: A Volume Detectionmentioning

confidence: 99%

“…In this regard, no compiler based tool is found to precisely analyze the complexity of programs. Instead, in literature profiling and statistical analysis are emphasized for estimating the time [28], [30], [31], [33], [35]. In the case of big data, one-time application profiling is not preferred, due to larger and complex datasets, which consumes time and resources resulting in substantial overhead [28]- [31], [33].…”

Section: B Velocity Detectionmentioning

confidence: 99%

“…Instead, in literature profiling and statistical analysis are emphasized for estimating the time [28], [30], [31], [33], [35]. In the case of big data, one-time application profiling is not preferred, due to larger and complex datasets, which consumes time and resources resulting in substantial overhead [28]- [31], [33]. In this situation, per machine regression analysis is a feasible option for predicting time [28], [31], [33], which is dependent on comprehensive training, and expected to be less accurate.…”

Section: B Velocity Detectionmentioning

confidence: 99%

“…These features can only be determined at run time, where the control flow information is available. Conventionally, the profiling technique has been used for determining code features, but it is discouraged for large scale applications due to substantial overhead [28]- [31]. Therefore, the proposed framework collects the feature set by first instrumenting each application, and then running it with smaller data sets.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Towards a Novel Framework for Automatic Big Data Detection

Ahmed

Ismail

2020

IEEE Access

View full text Add to dashboard Cite

Big data is a "relative" concept. It is the combination of data, application, and platform properties. Recently, big data specific technologies have emerged, including software frameworks, databases, hardware accelerators, storage technologies, etc. However, the automatic selection of these solutions for big data computations remains a non-trivial task. Presently, the big data tools are selected by analyzing the problem manually, or by using several performance prediction techniques. The manual identification is based on the data properties only, whereas the performance predictors only estimate basic execution metrics without linking them with big data (3Vs) thresholds. Hence, both ways of identification are mostly incorrect, which can lead to inefficient use of 3Vs optimizations, resulting into global inefficiency, reduced system performance, increasing power consumption, requiring greater effort on the part of the programming team, and misallocation of the hardware resources required for the task. In this regard, a novel framework has been proposed for automatic detection of 3Vs (Volume, Velocity, Variety) of big data, using machine learning. The detection is done through static code features, data, and platform properties, leading to relevant tool selection, and code generation, with minimal overheads, lesser programmer interventions, higher usability, and portability. Instead of handling each application with big data specialized solutions, or manually identifying the 3Vs, the framework can automatically detect and link the 3Vs to the relevant optimizations. Several standard applications have been tested using the proposed framework. In the case of volume, the average detection accuracy is up to 97.8% for seen and 95.9% for unseen applications. In the case of velocity, the average detection accuracy is up to 97.3% for seen and 92.6% for unseen applications. There is no margin of error in variety detection, as it has straightforward computations without any predictions. Furthermore, an airline recommendation system case study strengthens the effectiveness of the proposed approach.

show abstract

Section: A Volume Detectionmentioning

confidence: 99%