TLA: Topological Landscape Analysis for Single-Objective Continuous Optimization Problem Instances

Petelin, Gašper; Cenikj, Gjorgjina; Eftimov, Tome

doi:10.1109/ssci51031.2022.10022126

Cited by 8 publications

(7 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To provide an explanation for why this happens, Figure 5 presents the relation between the pairwise similarity of the ELA features representation (x-axis) and the pairwise difference in the ground truth performance of the optimization algorithm (y-axis) for the third problem in the CEC 2014, with the other problems. The heatmap shows that the third problem has four similar problem instances over 0.9 (17,21,29,30), as visible in Figure 5. In addition, we can see that the difference in ground algorithm performance of the problem and the similar instances is low, so the algorithm has similar behavior on these problems (see also Figure 1), and using them for the calibration helps to obtain better predictive errors.…”

Section: Resultsmentioning

confidence: 96%

“…Next, instead of exploratory landscape features calculated by a global sampling, we are planning to calculate them using the trajectory data that was observed by the algorithm during the run (i.e., to capture also information about the algorithm behavior) [14]. We are also going to try different problem feature representations such as topological data analysis [21]. Last but not least, we are planning to merge different benchmark suites to select representative problem instances [4,8] that will allow us to represent all possible landscape spaces from the problem space with the same number of problems that will further help the LOPO approach to have better prediction results.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

RF+clust for Leave-One-Problem-Out Performance Prediction

Nikolikj¹,

Doerr²,

Eftimov³

2023

Preprint

View full text Add to dashboard Cite

Per-instance automated algorithm configuration and selection are gaining significant moments in evolutionary computation in recent years. Two crucial, sometimes implicit, ingredients for these automated machine learning (AutoML) methods are 1) feature-based representations of the problem instances and 2) performance prediction methods that take the features as input to estimate how well a specific algorithm instance will perform on a given problem instance. Nonsurprisingly, common machine learning models fail to make predictions for instances whose feature-based representation is underrepresented or not covered in the training data, resulting in poor generalization ability of the models for problems not seen during training. In this work, we study leave-one-problem-out (LOPO) performance prediction. We analyze whether standard random forest (RF) model predictions can be improved by calibrating it with a weighted average of performance values obtained by the algorithm on problem instances that are sufficiently close to the problem for which a performance prediction is sought, measured by cosine similarity in feature space. While our RF+clust approach obtains more accurate performance prediction for several problems, its predictive power crucially depends on the chosen similarity threshold as well as on the feature portfolio for which the cosine similarity is measured, thereby opening a new angle for feature selection in a zero-shot learning setting, as LOPO is termed in machine learning.

show abstract

Section: Resultsmentioning

confidence: 96%

Section: Discussionmentioning

confidence: 99%

RF+clust for Leave-One-Problem-Out Performance Prediction

Nikolikj¹,

Doerr²,

Eftimov³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Our study demonstrated that performance prediction models built on ELA features effectively generalize across the three tested algorithms. Next, we will explore additional feature landscape meta-features, such as topological features [39] and those derived from deep neural network architectures [40], comparing them with ELA features to enhance predictive accuracy. Finally, we aim to evaluate these measures in an active learning setting, using them to determine if a model is suitable for new instances or if further training and fine-tuning are necessary.…”

Section: Discussionmentioning

confidence: 99%

“…In the future, we aim to identify specific meta-features tailored to individual algorithms or their respective families. This could involve conducting feature selection on the ELA features or creating and assessing alternative landscape features [39], [40].…”

Section: A First Experimentsmentioning

confidence: 99%

Assessing the Generalizability of a Performance Predictive Model

Nikolikj

Cenikj

Ispirova

et al. 2023

Proceedings of the Companion Conference on Genetic and Evolutionary Computation

View full text Add to dashboard Cite

A key component of automated algorithm selection and configuration, which in most cases are performed using supervised machine learning (ML) methods is a good-performing predictive model. The predictive model uses the feature representation of a set of problem instances as input data and predicts the algorithm performance achieved on them. Common machine learning models struggle to make predictions for instances with feature representations not covered by the training data, resulting in poor generalization to unseen problems. In this study, we propose a workflow to estimate the generalizability of a predictive model for algorithm performance, trained on one benchmark suite to another. The workflow has been tested by training predictive models across benchmark suites and the results show that generalizability patterns in the landscape feature space are reflected in the performance space.CCS Concepts: • Computing methodologies → Machine learning; Learning latent representations; Supervised learning; • Theory of computation → Design and analysis of algorithms.

show abstract

“…In [33], the authors provide an overview of the recent development of mathematical models and tools related to persistent homology, including software, feature representations, kernels, and similarity models in the context of machine learning. One approach is to use simplicial complexes, persistent homology, and other algebraic topology elements to gain better insight into the behavior of metaheuristic algorithms during their execution [34], and this approach is very successful. It is clear that the points in the fitness landscape that are visited during the execution of the metaheuristic algorithm (which change dynamically) can be a database (point cloud) for such a study.…”

Section: Computational Algebraic Topologymentioning

confidence: 99%

Topological Variable Neighborhood Search

Filipovic,

Kartelj

2024

Preprint

View full text Add to dashboard Cite

The design of the novel metaheuristic method, called Topological Variable Neighborhood Search, is presented and its theoretical properties are elaborated.The proposed metaheuristic method is implemented, applied to several well-known NP-hard problems on graphs (Metric Dimension Problem, Roman Domination Problem and Maximum Betweeness Problem) and compared with the relevant optimization methods for these problems. The obtained experimental results clearly show that the proposed method achieves a significant improvement for the investigated problems and outperforms other methods that participated in the comparison. In particular, it was shown that the Topological Variable Neighborhood Search is consistently better than the classical Variable Neighborhood Search.

show abstract

TLA: Topological Landscape Analysis for Single-Objective Continuous Optimization Problem Instances

Cited by 8 publications

References 19 publications

RF+clust for Leave-One-Problem-Out Performance Prediction

RF+clust for Leave-One-Problem-Out Performance Prediction

Assessing the Generalizability of a Performance Predictive Model

Topological Variable Neighborhood Search

Contact Info

Product

Resources

About