Proactive Dynamic Distributed Constraint Optimization Problems

Abstract

The Distributed Constraint Optimization Problem (DCOP) formulation is a powerful tool for modeling multi-agent coordination problems. To solve DCOPs in a dynamic environment, Dynamic DCOPs (D-DCOPs) have been proposed to model the inherent dynamism present in many coordination problems. D-DCOPs solve a sequence of static problems by reacting to changes in the environment as the agents observe them. Such reactive approaches ignore knowledge about future changes of the problem. To overcome this limitation, we introduce Proactive Dynamic DCOPs (PD-DCOPs), a novel formalism to model D-DCOPs in the presence of exogenous uncertainty. In contrast to reactive approaches, PD-DCOPs are able to explicitly model possible changes of the problem and take such information into account when solving the dynamically changing problem in a proactive manner. The additional expressivity of this formalism allows it to model a wider variety of distributed optimization problems. Our work presents both theoretical and practical contributions that advance current dynamic DCOP models: (i) We introduce Proactive Dynamic DCOPs (PD-DCOPs), which explicitly model how the DCOP will change over time; (ii) We develop exact and heuristic algorithms to solve PD-DCOPs in a proactive manner; (iii) We provide theoretical results about the complexity of this new class of DCOPs; and (iv) We empirically evaluate both proactive and reactive algorithms to determine the trade-offs between the two classes. The final contribution is important as our results are the first that identify the characteristics of the problems that the two classes of algorithms excel in.

Using Learning Automata to Solve Dynamic Distributed Constraint Optimization Problems

“…This method is based on automata learning in which agents can learn to decide correctly because multi-agent Q-learning methods do not guarantee convergence to achieve an optimal joint action. Moreover, based on our experimental results and related researches [9], if the scale of the problem increases in the previous methods mentioned in Table 2 not perform satisfactorily.…”

“…In final experiment, regarding convergence and runtime of our algorithm for different cases, we compare our results with baseline methods that are not using any learning technique in terms of run time and scaling. So we compare our suggested algorithm with the algorithms introduced in article [9] in term of run time and scaling. C-DPOP algorithm collapses the PD-DCOP into a DCOP and solves it with DPOP.…”

“…The Markovian DDCOPs algorithm [8] is a reactive online learning based on the joint action of agents. Unlike other methods, the proactive algorithms [9,27] consider the dependence between two consecutive time steps. In MA-DDCOP algorithms which are solved by RL techniques, learning of the values of variables by agents can be considered as an alternative to canonical DCOP solvers.…”

“…Researchers imagine dynamic DCOP as many sequences DCOP during time step and try to find optimized DCOP in each time step. In these researches, agents have holistic information about current environment (current DCOP) but are unaware of future changes of information [9]. So, ignores the dependence of each problem among each time steps and can't make use of information of previous DCOPs [8].…”

Proactive Agent Behaviour in Dynamic Distributed Constraint Optimisation Problems

“…While most extant DCOP studies in MAS focus on agent properties such as reactivity, learning, social abilities, scalability, and stabilisation, proactive agent behaviour is the goal of several approaches [13][14][15][16]. In proactive methods, an agent is expected to be goal-driven and not only react to environmental changes.…”