Automated Dynamic Algorithm Configuration

Adriaensen, Steven; Biedenkapp, André; Shala, Gresa; Awad, Noor H.; Eimer, Theresa; Lindauer, Marius; Hutter, Frank

doi:10.1613/jair.1.13922

Cited by 17 publications

(3 citation statements)

References 103 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Reinforcement learning excels at decision-making tasks, achieving a series of successes (Zhou, Li, and Wang 2020;Yang et al 2022;Haarnoja et al 2018;Wang et al 2022Wang et al , 2023bKuang et al 2022;Liu et al 2023;Zhou et al 2020) and finding increasing applications in specific tasks (Adriaensen et al 2022). We consider the stopping strategy as a reinforcement learning problem as well.…”

Section: Problem Formulationmentioning

confidence: 99%

Learning to Stop Cut Generation for Efficient Mixed-Integer Linear Programming

Ling,

Wang,

Wang

2024

AAAI

View full text Add to dashboard Cite

Cutting planes (cuts) play an important role in solving mixed-integer linear programs (MILPs), as they significantly tighten the dual bounds and improve the solving performance. A key problem for cuts is when to stop cuts generation, which is important for the efficiency of solving MILPs. However, many modern MILP solvers employ hard-coded heuristics to tackle this problem, which tends to neglect underlying patterns among MILPs from certain applications. To address this challenge, we formulate the cuts generation stopping problem as a reinforcement learning problem and propose a novel hybrid graph representation model (HYGRO) to learn effective stopping strategies. An appealing feature of HYGRO is that it can effectively capture both the dynamic and static features of MILPs, enabling dynamic decision-making for the stopping strategies. To the best of our knowledge, HYGRO is the first data-driven method to tackle the cuts generation stopping problem. By integrating our approach with modern solvers, experiments demonstrate that HYGRO significantly improves the efficiency of solving MILPs compared to competitive baselines, achieving up to 31% improvement.

show abstract

Section: Problem Formulationmentioning

confidence: 99%

Learning to Stop Cut Generation for Efficient Mixed-Integer Linear Programming

Ling,

Wang,

Wang

2024

AAAI

View full text Add to dashboard Cite

show abstract

“…Methods in AutoRL can be placed on a spectrum of automation, where on one end would be methods to select pipelines and on the other would be methods that try to discover new algorithms groundup in a data-driven manner (Oh et al, 2020). Techniques from the Automated Machine Learning literature (Hutter et al, 2019) then transfer to the RL setting, including algorithm selection (Laroche & Feraud, 2022), hyperparameter optimization (Li et al, 2019;Parker-Holder et al, 2020;Wan et al, 2022), dynamic configurations (Adriaensen et al, 2022), learned optimizers , and neural architecture search (Wan et al, 2022). Similarly, techniques from the Evolutionary optimization and Meta-Learning literature naturally transfer to this setting with methods aiming to meta-learn parts of the RL pipeline such as update rules (Oh et al, 2020), loss functions (Salimans et al, 2017;Kirsch et al, 2020), symbolic representations of algorithms (Alet et al, 2020;Co-Reyes et al, 2021;Luis et al, 2022), or concept drift (Lu et al, 2022).…”

Section: Automated Reinforcement Learning (Autorl)mentioning

confidence: 99%

Structure in Deep Reinforcement Learning: A Survey and Open Problems

Mohan,

Zhang,

Lindauer

2024

jair

View full text Add to dashboard Cite

Reinforcement Learning (RL), bolstered by the expressive capabilities of Deep Neural Networks (DNNs) for function approximation, has demonstrated considerable success in numerous applications. However, its practicality in addressing various real-world scenarios, characterized by diverse and unpredictable dynamics, noisy signals, and large state and action spaces, remains limited. This limitation stems from poor data efficiency, limited generalization capabilities, a lack of safety guarantees, and the absence of interpretability, among other factors. To overcome these challenges and improve performance across thesecrucial metrics, one promising avenue is to incorporate additional structural information about the problem into the RL learning process. Various sub-fields of RL have proposed methods for incorporating such inductive biases. We amalgamate these diverse methodologies under a unified framework, shedding light on the role of structure in the learning problem, and classify these methods into distinct patterns of incorporating structure. By leveraging this comprehensive framework, we provide valuable insights into the challenges of structured RL and lay the groundwork for a design pattern perspective on RL research. This novelperspective paves the way for future advancements and aids in developing more effective and efficient RL algorithms that can potentially handle real-world scenarios better.

show abstract

“…Dynamic Algorithm Configuration (DAC) offers an automated solution to the task of setting algorithm hyperparameters dynamically, by determining well-performing hyperparameter schedules or policies. One way to learn such policies is through Reinforcement Learning (RL) [1,3]. While conceptually appealing, RL algorithms have the notorious tendency to significantly overfit their training environments [14,15,19].…”

Section: Introductionmentioning

confidence: 99%

Instance Selection for Dynamic Algorithm Configuration with Reinforcement Learning: Improving Generalization

Benjamins,

Cenikj,

Nikolikj

et al. 2024

Proceedings of the Genetic and Evolutionary Computation Conference Companion

Self Cite

View full text Add to dashboard Cite

Dynamic Algorithm Configuration (DAC) addresses the challenge of dynamically setting hyperparameters of an algorithm for a diverse set of instances rather than focusing solely on individual tasks. Agents trained with Deep Reinforcement Learning (RL) offer a pathway to solve such settings. However, the limited generalization performance of these agents has significantly hindered the application in DAC. Our hypothesis is that a potential bias in the training instances limits generalization capabilities. We take a step towards mitigating this by selecting a representative subset of training instances to overcome overrepresentation and then retraining the agent on this subset to improve its generalization performance. For constructing the meta-features for the subset selection, we particularly account for the dynamic nature of the RL agent by computing time series features on trajectories of actions and rewards generated by the agent's interaction with the environment. Through empirical evaluations on the Sigmoid and CMA-ES benchmarks from the standard benchmark library for DAC, called DACBench, we discuss the potentials of our selection technique compared to training on the entire instance set. Our results highlight the efficacy of instance selection in refining DAC policies for diverse instance spaces. CCS CONCEPTS• Computing methodologies → Reinforcement learning; • Theory of computation → Design and analysis of algorithms.

show abstract

Automated Dynamic Algorithm Configuration

Cited by 17 publications

References 103 publications

Learning to Stop Cut Generation for Efficient Mixed-Integer Linear Programming

Learning to Stop Cut Generation for Efficient Mixed-Integer Linear Programming

Structure in Deep Reinforcement Learning: A Survey and Open Problems

Instance Selection for Dynamic Algorithm Configuration with Reinforcement Learning: Improving Generalization

Contact Info

Product

Resources

About