Fidelity metrics for estimation models

Javaid,; Ignjatovic,; Parameswaran, Sri

doi:10.1109/iccad.2010.5653959

Cited by 15 publications

(5 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to evaluate the fidelity of the estimated WCET we compute the rank correlation at each virtual unrolling factor using Kendall's τ correlation coefficient [16]. Rank correlation is a way to evaluate the fidelity of a ranking compared to a reference ranking.…”

Section: B Selection Of Virtual Unrolling Factormentioning

confidence: 99%

Reconciling Compiler Optimizations and WCET Estimation Using Iterative Compilation

Dardaillon¹,

Skalistis

Puaut

et al. 2019

2019 IEEE Real-Time Systems Symposium (RTSS)

View full text Add to dashboard Cite

Static Worst-Case Execution Time (WCET) estimation techniques operate upon the binary code of a program in order to provide the necessary input for schedulability analysis techniques. Compilers used to generate this binary code include tens of optimizations, that can radically change the flow information of the program. Such information is hard to be maintained across optimization passes and may render automatic extraction of important flow information, such as loop bounds, impossible. Thus, compiler optimizations, especially the sophisticated optimizations of mainstream compilers, are typically avoided. In this work, we explore for the first time iterativecompilation techniques that reconcile compiler optimizations and static WCET estimation. We propose a novel learning technique that selects sequences of optimizations that minimize the WCET estimate of a given program. We experimentally evaluate the proposed technique using an industrial WCET estimation tool (AbsInt aiT) over a set of 46 benchmarks from four different benchmarks suites, including reference WCET benchmark applications, image processing kernels and telecommunication applications. Experimental results show that WCET estimates are reduced on average by 20.3% using the proposed technique, as compared to the best compiler optimization level applicable.

show abstract

Section: B Selection Of Virtual Unrolling Factormentioning

confidence: 99%

Reconciling Compiler Optimizations and WCET Estimation Using Iterative Compilation

Dardaillon¹,

Skalistis

Puaut

et al. 2019

2019 IEEE Real-Time Systems Symposium (RTSS)

View full text Add to dashboard Cite

show abstract

“…3 shows that the performance ranking between methods strongly depends on noise type, training dataset and evaluation dataset. When calculating Kendall's Tau correlation coefficient [18], a value of 60% is obtained between Gaussian noise ranking and Interception noise ranking, ranking based on the mean PSNR. This correlation coefficient, while being high -Kendall's Tau is in [−1; 1]; −1 and 1 respectively meaning fully discordant and fully concordant rankings -, shows the need for a benchmark such as the proposed.…”

Section: Discussionmentioning

confidence: 99%

Opendenoising: An Extensible Benchmark for Building Comparative Studies of Image Denoisers

Lemarchand

Montesuma

Pelcat

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Image denoising has recently taken a leap forward due to machine learning. However, image denoisers, both expert-based and learning-based, are mostly tested on well-behaved generated noises (usually Gaussian) rather than on real-life noises, making performance comparisons difficult in real-world conditions. This is especially true for learning-based denoisers which performance depends on training data. Hence, choosing which method to use for a specific denoising problem is difficult.This paper proposes a comparative study of existing denoisers, as well as an extensible open tool that makes it possible to reproduce and extend the study. MWCNN is shown to outperform other methods when trained for a real-world image interception noise, and additionally is the second least compute hungry of the tested methods. To evaluate the robustness of conclusions, three test sets are compared. A Kendall's Tau correlation of only 60% is obtained on methods ranking between noise types, demonstrating the need for a benchmarking tool.

show abstract

“…A perfect Spearman correlation of +1 occurs when the variables are a perfect monotonically increasing function of each other. This property makes ρ a valid measure for fidelity [38]. there is no difference between the results of the roofline and the refined roofline model.…”

Section: E Evaluation For Test Setmentioning

confidence: 96%