Communication-Aware Path Design for Indoor Robots Exploiting Federated Deep Reinforcement Learning

Luo, Ruyu; Tian, Hui; Ni, Wanli

doi:10.1109/pimrc50174.2021.9569440

Cited by 10 publications

(6 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The maximum transmit power of the AP is P max = 20 dBm and the noise power spectral density is N 0 = −100 dBm/Hz. The channel model is the same as the settings in [20]. Other parameters are given in Table I.…”

Section: Numerical Resultsmentioning

confidence: 99%

Federated Deep Reinforcement Learning for RIS-Assisted Indoor Multi-Robot Communication Systems

Luo,

Ni,

Tian

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Indoor multi-robot communications face two key challenges: one is the severe signal strength degradation caused by blockages (e.g., walls) and the other is the dynamic environment caused by robot mobility. To address these issues, we consider the reconfigurable intelligent surface (RIS) to overcome the signal blockage and assist the trajectory design among multiple robots. Meanwhile, the non-orthogonal multiple access (NOMA) is adopted to cope with the scarcity of spectrum and enhance the connectivity of robots. Considering the limited battery capacity of robots, we aim to maximize the energy efficiency by jointly optimizing the transmit power of the access point (AP), the phase shifts of the RIS, and the trajectory of robots. A novel federated deep reinforcement learning (F-DRL) approach is developed to solve this challenging problem with one dynamic long-term objective. Through each robot planning its path and downlink power, the AP only needs to determine the phase shifts of the RIS, which can significantly save the computation overhead due to the reduced training dimension. Simulation results reveal the following findings: i) the proposed F-DRL can reduce at least 86% convergence time compared to the centralized DRL; ii) the designed algorithm can adapt to the increasing number of robots; iii) compared to traditional OMAbased benchmarks, NOMA-enhanced schemes can achieve higher energy efficiency.

show abstract

Section: Numerical Resultsmentioning

confidence: 99%

Federated Deep Reinforcement Learning for RIS-Assisted Indoor Multi-Robot Communication Systems

Luo,

Ni,

Tian

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…With the development of medical image processing and visualization technology, scholars have quantified the clinical needs and objectives of puncture surgery into multiple clinical conditions. Luo et al (2021) summarized the clinical conditions and divided them into two categories: strong constraints and weak constraints. Strong constraint is a rule that must be observed, and violation of it will lead to the failure of path planning.…”

Section: Multiple Needle Entry Points Selection Based On Strong and W...mentioning

confidence: 99%

Flexible needle puncture path planning for liver tumors based on deep reinforcement learning

Hu¹,

Jiang²,

Wang³

2022

Phys. Med. Biol.

View full text Add to dashboard Cite

Objective. Minimally invasive surgery has been widely adopted in the treatment of patients with liver tumors. In liver tumor puncture surgery, an image-guided ablation needle for puncture surgery, which first reaches a target tumor along a predetermined path, and then ablates the tumor or injects drugs near the tumor, is often used to reduce patient trauma, improving the safety of surgery operations and avoiding possible damage to large blood vessels and key organs. In this paper, a path planning method for computer tomography (CT) guided ablation needle in liver tumor puncture surgery is proposed. Approach. Given a CT volume containing abdominal organs, we first classify voxels and optimize the number of voxels to reduce volume rendering pressure, then we reconstruct a multi-scale 3D model of the liver and hepatic vessels. Secondly, multiple entry points of the surgical path are selected based on the strong and weak constraints of clinical puncture surgery through multi-agent reinforcement learning. We select the optimal needle entry point based on the length measurement. Then, through the incremental training of the double deep Q-learning network (DDQN), the transmission of network parameters from the small-scale environment to the larger-scale environment is accomplished, and the optimal surgical path with more optimized details is obtained. Main results. To avoid falling into local optimum in network training, improve both the convergence speed and performance of the network, and maximize the cumulative reward, we train the path planning network on different scales 3D reconstructed organ models, and validate our method on tumor samples from public datasets. The scores of human surgeons verified the clinical relevance of the proposed method. Significance. Our method can robustly provide the optimal puncture path of flexible needle for liver tumors, which is expected to provide a reference for surgeons' preoperative planning.

show abstract

“…Network Topology Design Objective Exploited DoF [3] Multi-hop Maximizing communication quality of service Mobility [4] Point-to-Point Minimizing the total energy consumption Mobility [5] Point-to-Point Minimizing the total energy consumption Mobility [6] Point-to-Point Maximizing the strength of receiving signal Mobility [7] Point-to-Point Maximizing the throughput guarantees to the video streaming Mobility [8] Multi-hop Maximizing the long-term throughput Mobility, Trans-mit Power [9] Multi-Unicast Maximizing the bandwidth-usage Topology [10] Multi-hop Maximizing the communication rate Transmit Power, Phase-shift [11] Point-to-Point Minimizing the total energy consumption Mobility [12] Multi-Unicast Minimizing the service discrepancy among all pairs of robots Mobility few studies that use some novel indicators, such as maximizing the throughput guarantees to the video streaming [7].…”

Section: Refmentioning

confidence: 99%

“…Equation (11) involves the joint optimization of the motion (position of the relay robot x r,k ) and communication (transmit powers of the relay robot P r,k and of the sensing robot P s,k ). Firstly, we formulate the same communication planning problem as in Equation ( 7) and the optimal transmit power is given by Equation (8). By substituting Equation (8) into Equation (11), we may transform the original problem into a motion planning problem, which is only relative to the positions x r = {x r,1 , x r,2 , .…”

Section: Jcmp Solutionsmentioning

confidence: 99%

“…There is an increasing interest in integrating autonomous robots into wireless communication networks [3][4][5][6][7][8][9][10][11][12][13], which is the method in which the robot utilizes its own degrees of freedom (mostly transmission power and mobility) to improve the communication quality of the system. For example, refs.…”

Section: Introduction and Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Joint Communication–Motion Planning in Networked Robotic Systems

Zhang

Zhang²,

Wu³

2022

Applied Sciences

View full text Add to dashboard Cite

In the recent decade, many research efforts in robotic society have considered motion planning for maintaining connectivity in networked robotic system (NRS) by exploiting robotic autonomous mobility. On the other hand, cognitive radio (CR) in the communication society aims at fully exploiting the spectrum in a wireless network, while the motion planning is seldom considered, as a wireless device itself may not decide where to go. In this article, joint communication–motion planning (JCMP) is proposed to boost the capability of NRS by exploiting both the adaptive communications and mobility control of autonomous robots. Specifically, we propose a JCMP framework for NRS, which aims at jointly exploiting the degree-of-freedom in mobility, space, time, frequency and power dimensions from both the motion and communication components. Afterward, we design and evaluate JCMP in a conventional and a CR-relay-assisted robot system, which shows the capability of JCMP in improving the performance of NRS. Finally, we summarize the proposed JCMP-enabled NRS framework and provide a series of future research directions.

show abstract

Communication-Aware Path Design for Indoor Robots Exploiting Federated Deep Reinforcement Learning

Cited by 10 publications

References 11 publications

Federated Deep Reinforcement Learning for RIS-Assisted Indoor Multi-Robot Communication Systems

Federated Deep Reinforcement Learning for RIS-Assisted Indoor Multi-Robot Communication Systems

Flexible needle puncture path planning for liver tumors based on deep reinforcement learning

Joint Communication–Motion Planning in Networked Robotic Systems

Contact Info

Product

Resources

About