Digital Twins-based Multi-agent Deep Reinforcement Learning for UAV-assisted Vehicle Edge Computing

Hu, Chen; Qi, Qi; Zhang, Lei; Liu, Cong; Chen, Dezhi; Liao, Jianxin; Zhuang, Zirui; Wang, Jingyu

doi:10.1109/smartworld-uic-atc-scalcom-digitaltwin-pricomp-metaverse56740.2022.00192

Cited by 6 publications

(2 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where σ j (1) represents the nonlinear response of the first layer, usually set to the ReLU function. The output of each layer in the network is fed as the input to the subsequent layer [47].…”

Section: A Mdp For the Stackelberg Game Between Rsus And Uavsmentioning

confidence: 99%

“…Unmanned Aerial Vehicles (UAVs) have garnered significant prominence in contemporary times owing to their multifaceted utility in expanding the scope of mobile communications, enhancing surveillance capabilities, and contributing to the development of intelligent urban environments [1]. The expeditious advancement of state-of-the-art technologies, encompassing blockchain, Artificial Intelligence (AI), and eXtended Reality (XR), coupled with the ubiquity of mobile devices, portends the emergence of metaverses as a transformative paradigm for the forthcoming generation [2].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Resource Allocation for Multi-Tenant Network Slicing: A Multi-Leader Multi-Follower Stackelberg Game Approach

Tran

2020

IEEE Trans. Veh. Technol.

View full text Add to dashboard Cite

The synergy between Unmanned Aerial Vehicles (UAVs) and metaverses is giving rise to an emerging paradigm named UAV metaverses, which create a unified ecosystem that blends physical and virtual spaces, transforming drone interaction and virtual exploration. UAV Twins (UTs), as the digital twins of UAVs that revolutionize UAV applications by making them more immersive, realistic, and informative, are deployed and updated on ground base stations, e.g., RoadSide Units (RSUs), to offer metaverse services for UAV Metaverse Users (UMUs). Due to the dynamic mobility of UAVs and limited communication coverages of RSUs, it is essential to perform real-time UT migration to ensure seamless immersive experiences for UMUs. However, selecting appropriate RSUs and optimizing the required bandwidth is challenging for achieving reliable and efficient UT migration. To address the challenges, we propose a tiny machine learning-based Stackelberg game framework based on pruning techniques for efficient UT migration in UAV metaverses. Specifically, we formulate a multi-leader multifollower Stackelberg model considering a new immersion metric of UMUs in the utilities of UAVs. Then, we design a Tiny Multi-Agent Deep Reinforcement Learning (Tiny MADRL) algorithm to obtain the tiny networks representing the optimal game solution. Specifically, the actor-critic network leverages the pruning techniques to reduce the number of network parameters and achieve model size and computation reduction, allowing for efficient implementation of Tiny MADRL. Numerical results demonstrate that our proposed schemes have better performance than traditional schemes.

show abstract

Section: A Mdp For the Stackelberg Game Between Rsus And Uavsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%