Benchmarks for Evaluating Optimization Algorithms and Benchmarking MATLAB Derivative-Free Optimizers for Practitioners’ Rapid Access

Lin, Li; Saldivar, Alfredo Alan Flores; Bai, Yun; Chen, Leo; Liu, Qunfeng; Li, Yun

doi:10.1109/access.2019.2923092

Cited by 14 publications

(9 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…, k! in the mathematical model (5). However, the existence of paradoxes does not depend on the exact values of P i , i = 1, .…”

Section: Occurrence Probabilities Of Paradoxesmentioning

confidence: 95%

“…The following matrix is another example of the probabilistic model (5), where three algorithms are tested on 50 problems and parameters p i satisfy condition (6):…”

Section: B "Survival Of the Nonfittest" Paradox: From "C2+"mentioning

confidence: 99%

“…, k! in the probabilistic model (5). From a theoretical viewpoint, the no-free-lunch (NFL) theorem in optimization [45] is suitable for our purpose.…”

Section: Occurrence Probabilities Of Paradoxesmentioning

confidence: 99%

“…N UMERICAL comparison has been widely used in testing and verifying the performance of optimization algorithms, especially, global optimization algorithms such as evolutionary algorithms [1]- [4]. Despite that a number of comparison methods and amalgamated scoring systems [5] exist, few studies have been reported on comparison strategies so far. In this article, we study comparison strategies and their mathematical properties.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Paradoxes in Numerical Comparison of Optimization Algorithms

Liu

Gehrlein

Yuan

et al. 2020

IEEE Trans. Evol. Computat.

Self Cite

View full text Add to dashboard Cite

Numerical comparison is often key to verifying the performance of optimization algorithms, especially, global optimization algorithms. However, studies have so far neglected issues concerning comparison strategies necessary to rank optimization algorithms properly. To fill this gap for the first time, we combine voting theory and numerical comparison research areas, which have been disjoint so far, and thus extend the results of the former to the latter for optimization algorithms. In particular, we investigate compatibility issues arising from comparing two and more than two algorithms, termed "C2" and "C2+" in this article, respectively. Through defining and modeling "C2" and "C2+" mathematically, it is uncovered and illustrated that numerical comparison can be incompatible. Further, two possible paradoxes, namely, "cycle ranking" and "survival of the nonfittest," are discovered and analyzed rigorously. The occurrence probabilities of these two paradoxes are also calculated under the no-free-lunch assumption, which shows the first justifiable use of the impartial culture assumption from voting theory, providing a point of reference to the frequency of the paradoxes occurring. It is also shown that significant influence on these probabilities comes from the number of algorithms and the number of optimization problems studied in the comparison. Further, various limiting probabilities when the number of optimization problems goes to infinity are also derived and characterized. The results would help guide benchmarking and developing optimization and machine learning algorithms.

show abstract

“…, k! in the mathematical model (5). However, the existence of paradoxes does not depend on the exact values of P i , i = 1, .…”

Section: Occurrence Probabilities Of Paradoxesmentioning

confidence: 95%

“…The following matrix is another example of the probabilistic model (5), where three algorithms are tested on 50 problems and parameters p i satisfy condition (6):…”

Section: B "Survival Of the Nonfittest" Paradox: From "C2+"mentioning

confidence: 99%

“…, k! in the probabilistic model (5). From a theoretical viewpoint, the no-free-lunch (NFL) theorem in optimization [45] is suitable for our purpose.…”

Section: Occurrence Probabilities Of Paradoxesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Paradoxes in Numerical Comparison of Optimization Algorithms

Liu

Gehrlein

Yuan

et al. 2020

IEEE Trans. Evol. Computat.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Salah satu upaya peningkatan mutu yaitu mengumpulkan data, analisis, observasi lapangan tetap sebagai acuan setiap penelitian.Untuk ukuran serta varian prosessor akan dipilih keluaran perusahaan Intel yang akan dibenchmark [15]. Adapun oranisasi avarge system performance yang telah melakukan pengujian terhadap 2 jenis varian generasi sebelumnya pada gambar 6.…”

Section: Pengumpulan Informasi Dan Dataunclassified

Analisis Performance Central Prosessing Unit (CPU) Realtime Menggunakan Metode Benchmarking

Yunanri¹,

Fauzan

Yani³

et al. 2021

matrik

View full text Add to dashboard Cite

Perkembangan teknologi semakin berkembang cepat baik dari performa, grafik, bandwidth dan lain-lainnya sehingga mempengaruhi berbagai sendi kehidupan dan profesi, hal ini menyebabkan perubahan sistem pada piranti atau kinerja pada central prosessing unit. Pada dunia bisnis, saat ini telah memfaatkan kemajuan teknologi informasi demi kelancaran kerja dibidang yang digeluti baik sekala kecil maupun sekala besar. Metode yang digunakan benchmarking merupakan suatu proses mengidetifikasi terhadap hardware dan proses suatu tolak ukur sebuah performa yang diharapkan. Adapun langkah pengujian melakukan evalusi kinerja central prosessing unit (CPU) yang dilakukan pada kinerja hardware atau perangkat keras baik prosessor, ram, vega dan lain sebagainya. Hasil pengujian yang dilaksanakan pada cental prosessing unit (CPU) penggunaan ram oleh prosessor i3 sebesar 3.1 Gb, GPU 3%, Disk uses 1%, penggunaan network atau jaringan 7.7 Mbps, penggunaan power suplay very low. Prosessor i5 sebesar 4.2 Gb, GPU 0%, Disk uses 0%, penggunaan network atau jaringan 7.7 Mbps, penggunaan power suplay low. Prosessor i7 sebesar 2.5 Gb, GPU 9%, Disk uses 9%, penggunaan network atau jaringan 104 Kbps, penggunaan power suplay high.

show abstract

A Bilevel Genetic Algorithm for Global Optimization Problems

Lun

Liu

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Benchmarks for Evaluating Optimization Algorithms and Benchmarking MATLAB Derivative-Free Optimizers for Practitioners’ Rapid Access

Cited by 14 publications

References 19 publications

Paradoxes in Numerical Comparison of Optimization Algorithms

Paradoxes in Numerical Comparison of Optimization Algorithms

Analisis Performance Central Prosessing Unit (CPU) Realtime Menggunakan Metode Benchmarking

A Bilevel Genetic Algorithm for Global Optimization Problems

Contact Info

Product

Resources

About