Multi-agent Markov decision processes with limited agent communication

Mukhopadhyay, Sudipta; Jain, B. B.

doi:10.1109/isic.2001.971476

Cited by 10 publications

(8 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We cast the joint optimization problem in (5) as a Multi-Agent Markov Decision Process (MMDP) [25] described as the tuple {S, A, P, R}, where S = S 1 × • • • × S N is a set of all possible states for all inXSs referred to as state space, A = A 1 × • • • × A N is the joint action space containing all possible actions (i.e., the set of all possible combinations of channels and power levels), R denotes the reward signal and P : S × A × S → ∆ is the transition function [25], where ∆ denotes the set of probability distributions over S.…”

Section: Resource Selection With Limited Informationmentioning

confidence: 99%

Multi-Agent Dynamic Resource Allocation in 6G in-X Subnetworks with Limited Sensing Information

Adeogun

Berardinelli

2022

Sensors

View full text Add to dashboard Cite

In this paper, we investigate dynamic resource selection in dense deployments of the recent 6G mobile in-X subnetworks (inXSs). We cast resource selection in inXSs as a multi-objective optimization problem involving maximization of the minimum capacity per inXS while minimizing overhead from intra-subnetwork signaling. Since inXSs are expected to be autonomous, selection decisions are made by each inXS based on its local information without signaling from other inXSs. A multi-agent Q-learning (MAQL) method based on limited sensing information (SI) is then developed, resulting in low intra-subnetwork SI signaling. We further propose a rule-based algorithm termed Q-Heuristics for performing resource selection based on similar limited information as the MAQL method. We perform simulations with a focus on joint channel and transmit power selection. The results indicate that: (1) appropriate settings of Q-learning parameters lead to fast convergence of the MAQL method even with two-level quantization of the SI, and (2) the proposed MAQL approach has significantly better performance and is more robust to sensing and switching delays than the best baseline heuristic. The proposed Q-Heuristic shows similar performance to the baseline greedy method at the 50th percentile of the per-user capacity and slightly better at lower percentiles. The Q-Heuristic method shows high robustness to sensing interval, quantization threshold and switching delay.

show abstract

Section: Resource Selection With Limited Informationmentioning

confidence: 99%

Multi-Agent Dynamic Resource Allocation in 6G in-X Subnetworks with Limited Sensing Information

Adeogun

Berardinelli

2022

Sensors

View full text Add to dashboard Cite

show abstract

“…Multi-agent MDP is a popular method for solving sequential optimization, decision making, and learning problems in an uncertain environment where the outcome depends on the previous actions (Mukhopadhyay & Jain, 2001). The presence of uncertainty regarding agent states and actions can lead to performance issues.…”

Section: Related Workmentioning

confidence: 99%

Cooperative solutions to exploration tasks under speed and budget constraints

Karishma

Rao

2022

Journal of Simulation

View full text Add to dashboard Cite

We present a multi-agent system where agents can cooperate to solve a system of dependent tasks, with agents having the capability to explore a solution space, make inferences, as well as query for information under a limited budget. Re-exploration of the solution space takes place by an agent when an older solution expires and is thus able to adapt to dynamic changes in the environment. We investigate the effects of task dependencies, with highly-dependent graph G 40 (a well-known program graph that contains 40 highly interlinked nodes, each representing a task) and less-dependent graphs G 18 (a program graph that contains 18 tasks with fewer links), increasing the speed of the agents and the complexity of the problem space and the query budgets available to agents. Specifically, we evaluate trade-offs between the agent's speed and query budget. During the experiments, we observed that increasing the speed of a single agent improves the system performance to a certain point only, and increasing the number of faster agents may not improve the system performance due to task dependencies. Favoring faster agents during budget allocation enhances the system performance, in line with the "Matthew effect." We also observe that allocating more budget to a faster agent gives better performance for a less-dependent system, but increasing the number of faster agents gives a better performance for a highly-dependent system.

show abstract

“…We cast the joint optimization problem in (5) as Multi-Agent Markov Decision Process (MMDP) [14] described as the tuple {S, A, P, R}, where S = S 1 × • • • × S N is a set of all possible states for all inXSs referred to as state space,…”

Section: Resource Selection With 1-bit Informationmentioning

confidence: 99%

“…the joint action space containing all possible actions (i.e., the set of all possible combinations of channels and power levels for problem I and all possible combinations of aggregated channels for problem II), R denotes the reward signal and P : S × A × S → ∆ is the transition function [14], where ∆ denotes the set of probability distributions over S.…”

Section: Resource Selection With 1-bit Informationmentioning

confidence: 99%

Intelligent Multi-Agent Resource Allocation in 6G in-X Subnetworks with Limited Sensing Information

Adeogun¹,

Berardinelli²

2022

Preprint

View full text Add to dashboard Cite

In this letter, we investigate dynamic resource selection in dense deployments of a recent 6G mobile in-X subnetworks (inXSs). We cast resource selection in inXSs as a multi-objective optimization problem involving maximization of per inXS sum capacities. Since inXSs are expected to be autonomous, selection decisions are made by each inXS based on its local information without signalling from other inXSs. A multi-agent Q-learning (MAQL) method based on limited sensing information (SI) is then developed resulting in significant reduction in the overhead associated with intra-subnetwork SI exchanges. We perform simulations with focus on two similar but distinct resource allocation problems: joint channel and transmit power selection and channel selection with aggregation. The results indicate that: 1) appropriate settings of Q-learning parameters leads to fast convergence of the MAQL method even with 1-bit quantization of the SI; 2) the proposed MAQL approach offer similar performance and is more robust to sensing delays than the best baseline heuristic with full SI.

show abstract

Multi-agent Markov decision processes with limited agent communication

Cited by 10 publications

References 5 publications

Multi-Agent Dynamic Resource Allocation in 6G in-X Subnetworks with Limited Sensing Information

Multi-Agent Dynamic Resource Allocation in 6G in-X Subnetworks with Limited Sensing Information

Cooperative solutions to exploration tasks under speed and budget constraints

Intelligent Multi-Agent Resource Allocation in 6G in-X Subnetworks with Limited Sensing Information

Contact Info

Product

Resources

About