USV Formation and Path-Following Control via Deep Reinforcement Learning With Random Braking

Zhao, Yujiao; Ma, Yong; Hu, Songlin

doi:10.1109/tnnls.2021.3068762

Cited by 119 publications

(28 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, the single point failure problem, which occurs when a leader does not work and the entire fleet cannot maintain a formation, can be avoided. There are three basic tasks for DL-based leaderless formation control in this part, i.e., path-guided [197]- [201], trajectory-guided [187], [202]- [208], and maneuvering-guided coordinated controls [209]- [213]. The DL model is generally applied to estimate the model uncertainties, i.e., to compensate for the unknown disturbance [197], [198], [204], [208]- [210], unknown dynamic [187], [197], [198], [200], [202]- [207], [209], [210], [212], unknown input coefficients [206], to solve the quadratic optimization problem [211], [213] or to generate the control signal [201].…”

Section: A Cooperative Controlmentioning

confidence: 99%

“…Path-guided Coordinated Controls: Based on different assumptions about what information is known or unknown by vehicles, existing research has made efforts to address unknown dynamics [197]- [199], unmodeled disturbances [197]- [199], [206], unknown kinetic models [200], input saturation [197], [199], velocity estimation [198], [200], communication reduction [197], [198], cyber-attack [199], system stability [197]- [199], and control signal generation [201].…”

Section: A Cooperative Controlmentioning

confidence: 99%

“…The DRL model is able to resolve the ASV formation path-following problem by designing reward functions that consider the velocity and error distance of each ASV related to the given formation [201]. ASVs are able to adjust their formation automatically and flexibly under the proposed DRLbased method.…”

Section: A Cooperative Controlmentioning

confidence: 99%

See 2 more Smart Citations

Survey of Deep Learning for Autonomous Surface Vehicles in the Marine Environment

Ye¹,

Yin²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Within the next several years, there will be a high level of autonomous technology that will be available for widespread use, which will reduce labor costs, increase safety, save energy, enable difficult unmanned tasks in harsh environments, and eliminate human error. Compared to software development for other autonomous vehicles, maritime software development, especially on aging but still functional fleets, is described as being in a very early and emerging phase. This introduces very large challenges and opportunities for researchers and engineers to develop maritime autonomous systems. Recent progress in sensor and communication technology has introduced the use of autonomous surface vehicles (ASVs) in applications such as coastline surveillance, oceanographic observation, multivehicle cooperation, and search and rescue missions. Advanced artificial intelligence technology, especially deep learning (DL) methods that conduct nonlinear mapping with self-learning representations, has brought the concept of full autonomy one step closer to reality. This paper surveys the existing work regarding the implementation of DL methods in ASV-related fields. First, the scope of this work is described after reviewing surveys on ASV developments and technologies, which draws attention to the research gap between DL and maritime operations. Then, DL-based navigation, guidance, control (NGC) systems and cooperative operations, are presented. Finally, this survey is completed by highlighting the current challenges and future research directions.

show abstract

Section: A Cooperative Controlmentioning

confidence: 99%

Section: A Cooperative Controlmentioning

confidence: 99%

See 1 more Smart Citation

Survey of Deep Learning for Autonomous Surface Vehicles in the Marine Environment

Ye¹,

Yin²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The control policy performed well in the real drone flight test. Zhao et al [21] addressed the problem of path following for underactuated USV formation via a changed DRL with random braking. With the aid of DRL, the proposed system could adjust the formation automatically and flexibly.…”

Section: Introductionmentioning

confidence: 99%

A Deep $Q$ -Network-Based Collaborative Control Research for Smart Ammunition Formation

Shen

Zhang

Zhu

et al. 2022

International Journal of Aerospace Engineering

View full text Add to dashboard Cite

The smart ammunition formation (SAF) system model usually has the characteristics of complexity, time variation, and nonlinearity. With the consideration of random factors, such as sensor error and environmental disturbance, the system model cannot be modeled accurately. To deal with this problem, this paper investigated an intelligent deep Q -network- (DQN-) based control algorithm for the SAF collaborative control, which deals with the high dynamics and uncertainty in the SAF flight environment. In the environment description of the SAF, we built a dynamic model to represent the system joint states, which referred to the smart ammunition’s velocity, the trajectory inclination angle, the ballistic deflection angle, and the relative position between different formation nodes. Next, we describe the SAF collaborative control process as a Markov decision process (MDP) with the application of the reinforcement learning (RL) technique. Then, the basic framework ε -imitation action-selecting strategy and the algorithm details were developed to address the SAF control problem based on the DQN scheme. Finally, the numerical simulation was carried out to verify the effectiveness and portability of the DQN-based algorithm. The average total reward curve showed a reasonable convergence, and the relative kinematic relationship among the formation nodes met the requirements of the controller design. It illustrated that the DQN-based algorithm obtained a novel performance in the SAF collaborative control.

show abstract

“…Due to the limitations of a single USV operation, a single USV can not efficiently complete or even complete complex tasks, with a small operating range and poor fault tolerance. Scholars are currently conducting extensive research on the performance of USV formations in some special missions [1]- [5]. Multiple USVs operating at the same time can enable marine operations to be more durable, intelligent and large-scale, becoming one of the important development trends of future marine operations.…”

Section: Introductionmentioning

confidence: 99%

Back-Stepping Formation Control of Unmanned Surface Vehicles With Input Saturation Based on Adaptive Super-Twisting Algorithm

et al. 2022

View full text Add to dashboard Cite

Aiming at the formation control problem for unmanned surface vehicles under input overload and external disturbance, a leader-follower formation control law based on input saturation and the adaptive super-twisting algorithm was designed in this paper. Firstly, the mathematical model of underactuated unmanned surface vehicle formation based on the leader-follower method is established, and the virtual expected velocity is designed by the back-stepping method to improve the control accuracy of formation. Secondly, the parametric uncertainties and unknown external disturbances in the USV's dynamical model are compensated by the proposed adaptive super-twisting control laws. In addition, the input saturation function is added to the controller to avoid machine necrosis caused by input overload. Finally, Simulation results show that the controller can roughly keep the trajectory of the USV consistent with the expected trajectory and ensure that the input values are within the safe range, which proves the effectiveness of the method in this paper.INDEX TERMS unmanned surface vehicles, multi-agent system, super-twisting algorithm, input saturation.

show abstract

USV Formation and Path-Following Control via Deep Reinforcement Learning With Random Braking

Cited by 119 publications

References 28 publications

Survey of Deep Learning for Autonomous Surface Vehicles in the Marine Environment

Survey of Deep Learning for Autonomous Surface Vehicles in the Marine Environment

A Deep $Q$ -Network-Based Collaborative Control Research for Smart Ammunition Formation

Back-Stepping Formation Control of Unmanned Surface Vehicles With Input Saturation Based on Adaptive Super-Twisting Algorithm

Contact Info

Product

Resources

About

USV Formation and Path-Following Control via Deep Reinforcement Learning With Random Braking

Cited by 119 publications

References 28 publications

Survey of Deep Learning for Autonomous Surface Vehicles in the Marine Environment

Survey of Deep Learning for Autonomous Surface Vehicles in the Marine Environment

A Deep Q -Network-Based Collaborative Control Research for Smart Ammunition Formation

Back-Stepping Formation Control of Unmanned Surface Vehicles With Input Saturation Based on Adaptive Super-Twisting Algorithm

Contact Info

Product

Resources

About

A Deep $Q$ -Network-Based Collaborative Control Research for Smart Ammunition Formation