A Reinforcement Learning Approach for Interdomain Routing with Link Prices

Vrancx, Peter; Gurzi, Pasquale; Rodriguez, Abdel; Steenhaut, Kris; Nowé, Ann

doi:10.1145/2719648

Cited by 3 publications

(4 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a final remark, our goal is not to propose a new interdomain routing model, or infer more accurately the routing policies in the Internet, but to pinpoint the intersections and disparities of the results our replication effort against the findings and insights of the original work [63]. Finally, we believe that our results can aid in the understanding of a variety of interdomain routing applications, such as the measurement of the RPKI adoption [17,49], fine-grained interdomain policy learning [62,67], interdomain routing verification [10], privacy-preserving routing [13], discovering caching policies in the wild [18,34] and studying routing attacks [56].…”

Section: Introductionmentioning

confidence: 83%

Replication: 20 Years of Inferring Interdomain Routing Policies

Kastanakis,

Giotsas,

Livadariu

et al. 2023

Proceedings of the 2023 ACM on Internet Measurement Conference

View full text Add to dashboard Cite

In 2003, Wang and Gao [63] presented an algorithm to infer and characterize routing policies as this knowledge could be valuable in predicting and debugging routing paths. They used their algorithm to measure the phenomenon of selectively announced prefixes, in which, ASes would announce their prefixes to specific providers to manipulate incoming traffic. Since 2003, the Internet has evolved from a hierarchical graph, to a flat and dense structure. Despite 20 years of extensive research since that seminal work, the impact of these topological changes on routing policies is still blurred.In this paper we conduct a replicability study of the Wang and Gao paper [63], to shed light on the evolution and the current state of selectively announced prefixes. We show that selective announcements are persistent, not only across time, but also across networks. Moreover, we observe that neighbors of different AS relationships may be assigned with the same local preference values, and path selection is not as heavily dependent on AS relationships as it used to be. Our results highlight the need for BGP policy inference to be conducted as a high-periodicity process to account for the dynamic nature of AS connectivity and the derived policies.

show abstract

Section: Introductionmentioning

confidence: 83%

Replication: 20 Years of Inferring Interdomain Routing Policies

Kastanakis,

Giotsas,

Livadariu

et al. 2023

Proceedings of the 2023 ACM on Internet Measurement Conference

View full text Add to dashboard Cite

show abstract

“…Learning agents in these studies include robots [26]- [33], smarts sensors [34]- [41] and smart grid elements [42]- [45]. Network-based applications are the second most explored domain with network controllers [46], [47] or network nodes [48]- [53] being the learning agents. CPS and network-based applications are employed in more than half of the studies (53%).…”

Section: A Rq1: Csas Characteristics 1) Application Domain and Agentsmentioning

confidence: 99%

“…In [59]- [61], agents learn individually how to play the specified games and cooperate to achieve a positive outcome, i.e., win the game. Similarly, within the network domain, agents individually learn to propagate messages [50], predict the loss [48], and other individual tasks [46], [49], [53]. In this context, the emergent CSAS behaviour is the systemwide optimal dissemination of information.…”

Section: B Rq2: Learning Purposementioning

confidence: 99%

On Learning in Collective Self-Adaptive Systems: State of Practice and a 3D Framework

D'Angelo

Gerasimou

Ghahremani

et al. 2019

2019 IEEE/ACM 14th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS)

View full text Add to dashboard Cite

Collective self-adaptive systems (CSAS) are distributed and interconnected systems composed of multiple agents that can perform complex tasks such as environmental data collection, search and rescue operations, and discovery of natural resources. By providing individual agents with learning capabilities, CSAS can cope with challenges related to distributed sensing and decision-making and operate in uncertain environments. This unique characteristic of CSAS enables the collective to exhibit robust behaviour while achieving system-wide and agent-specific goals. Although learning has been explored in many CSAS applications, selecting suitable learning models and techniques remains a significant challenge that is heavily influenced by expert knowledge. We address this gap by performing a multifaceted analysis of existing CSAS with learning capabilities reported in the literature. Based on this analysis, we introduce a 3D framework that illustrates the learning aspects of CSAS considering the dimensions of autonomy, knowledge access, and behaviour, and facilitates the selection of learning techniques and models. Finally, using example applications from this analysis, we derive open challenges and highlight the need for research on collaborative, resilient and privacy-aware mechanisms for CSAS.

show abstract

“…Recently, an application for interdomain routing used an exploration for adapting to changing environments [49]. Vrancx et al refer to this exploration as selective exploration and it is based on computing a probability distribution of the past rewards and when the most recent one falls far away from the distribution a random action to explore is triggered.…”

Section: Multiagent Explorationmentioning

confidence: 99%

An exploration strategy for non-stationary opponents

Hernández-Leal

Zhan

Taylor

et al. 2016

Auton Agent Multi-Agent Syst

View full text Add to dashboard Cite

The success or failure of any learning algorithm is partially due to the exploration strategy it exerts. However, most exploration strategies assume that the environment is stationary and non-strategic. In this work we shed light on how to design exploration strategies in non-stationary and adversarial environments. Our proposed adversarial drift exploration (DE) is able to efficiently explore the state space while keeping track of regions of the environment that have changed. This proposed exploration is general enough to be applied in single agent non-stationary environments as well as in multiagent settings where the opponent changes its strategy in time. We use a two agent strategic interaction setting to test this new type of exploration, where the opponent switches between different behavioral patterns to emulate a non-deterministic, stochastic and adversarial environment. The agent's objective is to learn a model of the opponent's strategy to act optimally. Our contribution is twofold. First, we present DE as a strategy for switch detection. Second, we propose a new algorithm Most of this work was performed while the first author was a graduate student at INAOE. This paper extends the paper "Exploration strategies to detect strategy switches" presented at the Adaptive Learning Agents workshop [27]. 123Auton Agent Multi-Agent Syst called R-max# for learning and planning against non-stationary opponent. To handle such opponents, R-max# reasons and acts in terms of two objectives: (1) to maximize utilities in the short term while learning and (2) eventually explore opponent behavioral changes. We provide theoretical results showing that R-max# is guaranteed to detect the opponent's switch and learn a new model in terms of finite sample complexity. R-max# makes efficient use of exploration experiences, which results in rapid adaptation and efficient DE, to deal with the non-stationary nature of the opponent. We show experimentally how using DE outperforms the state of the art algorithms that were explicitly designed for modeling opponents (in terms average rewards) in two complimentary domains.

show abstract

A Reinforcement Learning Approach for Interdomain Routing with Link Prices

Cited by 3 publications

References 23 publications

Replication: 20 Years of Inferring Interdomain Routing Policies

Replication: 20 Years of Inferring Interdomain Routing Policies

On Learning in Collective Self-Adaptive Systems: State of Practice and a 3D Framework

An exploration strategy for non-stationary opponents

Contact Info

Product

Resources

About