Reinforcement Learning-Based Formation Pinning and Shape Transformation for Swarms

Dong, Zhaoqi; Wu, Qizhen; Chen, Lei

doi:10.3390/drones7110673

Cited by 4 publications

(4 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to differentiate the surprising and boring transitions, PER uses the temporal difference (TD) error that simply computes the difference between target network prediction and q-network prediction to assign a priority over the received experiences [53]. A big TD error results in a higher priority:…”

Section: Prioritized Experience Replaymentioning

confidence: 99%

Genetic-Algorithm-Aided Deep Reinforcement Learning for Multi-Agent Drone Delivery

Tarhan,

Ure

2024

Drones

View full text Add to dashboard Cite

The popularity of commercial unmanned aerial vehicles has drawn great attention from the e-commerce industry due to their suitability for last-mile delivery. However, the organization of multiple aerial vehicles efficiently for delivery within limitations and uncertainties is still a problem. The main challenge of planning is scalability, since the planning space grows exponentially to the number of agents, and it is not efficient to let human-level supervisors structure the problem for large-scale settings. Algorithms based on Deep Q-Networks had unprecedented success in solving decision-making problems. Extension of these algorithms to multi-agent problems is limited due to scalability issues. This work proposes an approach that improves the performance of Deep Q-Networks on multi-agent delivery by drone problems by utilizing state decompositions for lowering the problem complexity, Curriculum Learning for handling the exploration complexity, and Genetic Algorithms for searching efficient packet-drone matching across the combinatorial solution space. The performance of the proposed method is shown in a multi-agent delivery by drone problem that has 10 agents and ≈1077 state–action pairs. Comparative simulation results are provided to demonstrate the merit of the proposed method. The proposed Genetic-Algorithm-aided multi-agent DRL outperformed the rest in terms of scalability and convergent behavior.

show abstract

Section: Prioritized Experience Replaymentioning

confidence: 99%

Genetic-Algorithm-Aided Deep Reinforcement Learning for Multi-Agent Drone Delivery

Tarhan,

Ure

2024

Drones

View full text Add to dashboard Cite

show abstract

“…To solve the optimization problem, it is proposed to use methods of multicriteria analysis of decision making problems [12][13] and the criteria importance theory [14][15][16][17], in which the concepts of criteria importance are formally defined.…”

Section: Multicriteria Decision Makingmentioning

confidence: 99%

“…Using quantitative estimates of the importance coefficients 𝛼𝛼 𝑖𝑖 and values of gradations 𝑣𝑣(𝑘𝑘), an additive value function can be constructed [12][13]15]:…”

Section: Multicriteria Decision Makingmentioning

confidence: 99%

“…It addresses problems related to simulating the emergent behavior of autonomous agents, such as alignment, cohesion, and repulsion, to imitate natural flocking movements. In [12] a method is considered that gives to enable drone swarms to quickly and effectively adapt to dynamic external environments. It is based on the Q-learning network to improve the cohesion and repulsion parameters in the Boids model to achieve continuous obstacle avoidance and maximize spatial coverage in the simulation scenario.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multicriteria adaptation of agricultural spraying drones

Misyurin,

Neluybin,

Nosova

et al. 2024

BIO Web Conf.

View full text Add to dashboard Cite

The article proposes a method for searching for the best configurations of robots under new and previously unknown operating conditions, which makes it possible to find solutions close to optimal using a limited number of full-scale tests. The problem of processing agricultural crops using liquid-spraying UAVs is considered. The time required for a given UAV to process a given landing area under arbitrarily specified external conditions was selected as a performance indicator of the UAV configuration. The search algorithm is based on multicriteria analysis of vector characteristics of UAVs using the criteria importance theory. Previous studies have used information about the relative importance of criteria. In this work, we additionally evaluate the value of gradations of the criteria scale and construct a value function for UAV configurations. Using a simple simulation model, a numerical experiment was carried out for 1000 different operating conditions. The effectiveness of the proposed method is confirmed by the small size of deviations of the found solutions from the optimal ones.

show abstract

Digital Twin‐Enabled Deep Reinforcement Learning for Safety‐Guaranteed Flocking Motion of UAV Swarm

Li,

Lei,

Shen

et al. 2024

Trans Emerging Tel Tech

View full text Add to dashboard Cite

Multi‐agent deep reinforcement learning (MADRL) has become a typical paradigm for the flocking motion of UAV swarm in dynamic, stochastic environments. However, sim‐to‐real problems, such as reality gap, training efficiency, and safety issues, restrict the application of MADRL in flocking motion scenarios. To address these problems, we first propose a digital twin (DT)‐enabled training framework. With the assistance of high‐fidelity digital twin simulation, effective policies can be efficiently trained. Based on the multi‐agent proximal policy optimization (MAPPO) algorithm, we then design the learning approach for flocking motion with matching observation space, action space, and reward function. Afterward, we employ a distributed flocking center estimation algorithm based on position consensus. The estimated center is used as a policy input to improve the aggregation behavior. Moreover, we introduce a repulsion scheme, which applies an additional repulsion force to the action to prevent UAVs from colliding with neighbors and obstacles. Simulation results show that our method performs well in maintaining flocking formation and avoiding collisions, and has better decision‐making ability in near‐realistic environments.

show abstract

Reinforcement Learning-Based Formation Pinning and Shape Transformation for Swarms

Cited by 4 publications

References 36 publications

Genetic-Algorithm-Aided Deep Reinforcement Learning for Multi-Agent Drone Delivery

Genetic-Algorithm-Aided Deep Reinforcement Learning for Multi-Agent Drone Delivery

Multicriteria adaptation of agricultural spraying drones

Digital Twin‐Enabled Deep Reinforcement Learning for Safety‐Guaranteed Flocking Motion of UAV Swarm

Contact Info

Product

Resources

About