No-Regret Learning in Dynamic Stackelberg Games

Lauffer, Niklas; Ghasemi, Mahsa; Hashemi, Abolfazl; Savas, Yagiz; Topcu, Ufuk

doi:10.48550/arxiv.2202.04786

Cited by 3 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Stackelberg games. Our problem can be viewed as an online learning version of repeated Stackelberg game (Von Stackelberg 2010; Marecki, Tesauro, and Segal 2012;Bai et al 2021;Lauffer et al 2022;Zhao et al 2023). A common objective in this area of work is to minimize a Stackelberg regret, i.e., difference to the optimal policy that knows the leader's optimal action in hindsight, and the above works aim to minimize the cumulative Stackelberg regret of a leader, assuming that a follower best responds at each round.…”

Section: Related Workmentioning

confidence: 99%

Regret Analysis of Repeated Delegated Choice

Hajiaghayi,

Mahdavi,

Rezaei

et al. 2024

AAAI

View full text Add to dashboard Cite

We present a study on a repeated delegated choice problem, which is the first to consider an online learning variant of Kleinberg and Kleinberg, EC'18. In this model, a principal interacts repeatedly with an agent who possesses an exogenous set of solutions to search for efficient ones. Each solution can yield varying utility for both the principal and the agent, and the agent may propose a solution to maximize its own utility in a selfish manner. To mitigate this behavior, the principal announces an eligible set which screens out a certain set of solutions. The principal, however, does not have any information on the distribution of solutions nor the number of solutions in advance. Therefore, the principal dynamically announces various eligible sets to efficiently learn the distribution. The principal's objective is to minimize cumulative regret compared to the optimal eligible set in hindsight. We explore two dimensions of the problem setup, whether the agent behaves myopically or strategizes across the rounds, and whether the solutions yield deterministic or stochastic utility. We obtain sublinear regret upper bounds in various regimes, and derive corresponding lower bounds which implies the tightness of the results. Overall, we bridge a well-known problem in economics to the evolving area of online learning, and present a comprehensive study in this problem.

show abstract

Section: Related Workmentioning

confidence: 99%

Regret Analysis of Repeated Delegated Choice

Hajiaghayi,

Mahdavi,

Rezaei

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…In learningbased control, Lyapunov theory, model predictive control, and control barrier functions are also employed to develop robust learning algorithms (Choi et al 2020;Zheng et al 2021;Cheng et al 2019;Ames et al 2016;Berkenkamp et al 2017;Sun, Kim, and How 2021;Chriat and Sun 2023b,c,a;Kanellopoulos et al 2021). Additionally, with the worst-case criterion for safety, minimax policy optimization (Li et al 2019;Zhang, Yang, and Basar 2019) or its generalization Stackelberg games (Yang et al 2022;Zhou and Xu 2021;Lauffer et al 2022;Bai et al 2021) are often the frameworks to promote resilience. Other works include meta-adaptive nonlinear control integrating learning modules for fast adaptation in unpredictable settings (Shi et al 2021;O'Connell et al 2022).…”

Section: Related Workmentioning

confidence: 99%

Constrained Meta-Reinforcement Learning for Adaptable Safety Guarantee with Differentiable Convex Programming

Cho,

Sun

2024

AAAI

View full text Add to dashboard Cite

Despite remarkable achievements in artificial intelligence, the deployability of learning-enabled systems in high-stakes real-world environments still faces persistent challenges. For example, in safety-critical domains like autonomous driving, robotic manipulation, and healthcare, it is crucial not only to achieve high performance but also to comply with given constraints. Furthermore, adaptability becomes paramount in non-stationary domains, where environmental parameters are subject to change. While safety and adaptability are recognized as key qualities for the new generation of AI, current approaches have not demonstrated effective adaptable performance in constrained settings. Hence, this paper breaks new ground by studying the unique challenges of ensuring safety in nonstationary environments by solving constrained problems through the lens of the meta-learning approach (learning to learn). While unconstrained meta-learning already encounters complexities in end to end differentiation of the loss due to the bi-level nature, its constrained counterpart introduces an additional layer of difficulty, since the constraints imposed on task-level updates complicate the differentiation process. To address the issue, we first employ successive convex-constrained policy updates across multiple tasks with differentiable convex programming, which allows meta-learning in constrained scenarios by enabling end-to-end differentiation. This approach empowers the agent to rapidly adapt to new tasks under nonstationarity while ensuring compliance with safety constraints. We also provide a theoretical analysis demonstrating guaranteed monotonic improvement of our approach, justifying our algorithmic designs. Extensive simulations across diverse environments provide empirical validation with significant improvement over established benchmarks.

show abstract