Predicting Computer Performance Based on Hardware Configuration Using Multiple Neural Networks

López, Luciano Espino; Guynn, Michael; Lu, Meiliu

doi:10.1109/icmla.2018.00132

Cited by 7 publications

(5 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Before we continue further and present the final models, let us make a decision about the solver (optimiser) of our final MLP model by considering neural networks used in related work. Based on the models developed in previous studies, we explored adam models with three hidden layers of (12,12,12), (16,16,16) [2], (50, 100, 50) [24], and (100, 100, 100) [3] with the maximum 10000 iterations (which determines the number of epochs), and early stopping enabled. The SPEC datasets, the development frameworks, and the exact features used in these studies are not the same as ours, but we have attempted to replicate their models as closely as possible.…”

Section: ) Mlp Discussion and Final Modelsmentioning

confidence: 99%

See 1 more Smart Citation

Comparative Analysis of Machine Learning Models for Performance Prediction of the SPEC Benchmarks

Tousi

Luján

2022

IEEE Access

View full text Add to dashboard Cite

Simulation-based performance prediction is complicated and time-consuming. In this study, we apply supervised learning to predict the performance scores of Standard Performance Evaluation Corporation (SPEC) benchmarks. The SPEC CPU2017 is a public dataset of results obtained by executing 43 standardised performance benchmarks organised into 4 suites on various system configurations. This paper analyses the dataset and aims to answer the following questions: I) can we accurately predict the SPEC results based on the configurations provided in the dataset, without having to actually run the benchmarks? II) what are the most important hardware and software features? III) what are the best predictive models and hyperparameters, in terms of prediction error and time? and IV) can we predict the performance of future systems using the past data? We present how to prepare data, select features, tune hyperparameters and evaluate regression models based on Multi-Task Elastic-Net, Decision Tree, Random Forest, and Multi-Layer Perceptron neural networks estimators. Feature selection is performed in three steps: removing zero variance features, removing highly correlated features, and Recursive Feature Elimination based on different feature importance metrics: elastic-net coefficients, tree-based importance measures and Permutation Importance. We select the best models using grid search on the hyperparameter space, and finally, compare and evaluate the performance of the models. We show that tree-based models with the original 29 features provide accurate predictions with an average error of less than 4%. The average error of faster Decision Tree and Random Forest models with 10 features is still below 6% and 5% respectively.

show abstract

Section: ) Mlp Discussion and Final Modelsmentioning

confidence: 99%

“…Also, the memory-intensive rate benchmarks do not scale well, as even the highest-spec machines cannot meet their memory bandwidth requirements. Authors in [24] have used multiple neural networks to predict the performance of a machine, i.e. a hardware configuration.…”

Section: Related Workmentioning

confidence: 99%

Comparative Analysis of Machine Learning Models for Performance Prediction of the SPEC Benchmarks

Tousi

Luján

2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Technique(s) Prediction [19] SPEC CPU / SPEC Java Server Custom linear regression model Server benchmark performances [25] SPEC 2006 Custom linear regression model Performance of future systems [9] SPEC CPU2000 / CPU2006 Hybrid mechanistic-empirical model Commercial processor performance [15] SPEC OpenMP Classic fractal-based sampling Accelerating multithreaded app simulation [35] SPEC 2006 Fine-grained phase-based approach Performance and power [23]…”

Section: Work Dataset(s)mentioning

confidence: 99%

“…Lopez et al [23] used multiple neural networks for a classification task for predicting the best computer hardware configuration options. Although their work demonstrates the validity of using Deep Learning on SPEC datasets, their underlying problem is quite different to ours.…”

Section: Predictions From the Spec Datasetsmentioning

confidence: 99%

Predicting the Performance of a Computing System with Deep Networks

Cengiz

Forshaw

Atapour-Abarghouei

et al. 2023

Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering

View full text Add to dashboard Cite

Predicting the performance and energy consumption of computing hardware is critical for many modern applications. This will inform procurement decisions, deployment decisions, and autonomic scaling. Existing approaches to understanding the performance of hardware largely focus around benchmarking -leveraging standardised workloads which seek to be representative of an end-user's needs. Two key challenges are present; benchmark workloads may not be representative of an end-user's workload, and benchmark scores are not easily obtained for all hardware. Within this paper, we demonstrate the potential to build Deep Learning models to predict benchmark scores for unseen hardware. We undertake our evaluation with the openly available SPEC 2017 benchmark results. We evaluate three different networks, one fully-connected network along with two Convolutional Neural Networks (one bespoke and one ResNet inspired) and demonstrate impressive 𝑅 2 scores of 0.96, 0.98 and 0.94 respectively. CCS CONCEPTS• Computing methodologies → Neural networks.

show abstract

“…Hardware performance prediction is a well-studied topic. Lopez et al [8] explored a way to predict computer performance based on hardware component data without needing simulation. They used a deep learning model to generate a benchmark score for a given hardware configuration, then used multiple neural networks and principal component analysis to predict performance in comparison to the corresponding benchmarks.…”

Section: Related Workmentioning

confidence: 99%

Untitled

2020

IJAIA

View full text Add to dashboard Cite

We propose a set of methods to classify vendors based on estimated CPU performance and predict CPU performance based on hardware components. For vendor classification, we use the highest and lowest estimated performance and frequency of occurrences of each vendor to create classification zones. These zones can be used to identify vendors who manufacture hardware that satisfy a given performance requirement. We use multi-layered neural networks for performance prediction, which account for nonlinearity in performance data. Various neural network architectures are analysed in comparison to linear, quadratic, and cubic regression. Experiments show that neural networks obtain low error and high correlation between predicted and published performance values, while cubic regression produces higher correlation than neural networks when more data is used for training than testing. An analysis of how the neural network architecture affects prediction is also performed. The proposed methods can be used to identify suitable hardware replacements.

show abstract

Predicting Computer Performance Based on Hardware Configuration Using Multiple Neural Networks

Cited by 7 publications

References 12 publications

Comparative Analysis of Machine Learning Models for Performance Prediction of the SPEC Benchmarks

Comparative Analysis of Machine Learning Models for Performance Prediction of the SPEC Benchmarks

Predicting the Performance of a Computing System with Deep Networks

Untitled

Contact Info

Product

Resources

About