Autonomous Driving Control Using the DDPG and RDPG Algorithms

Chang, Che-Cheng; Tsai, Jichiang; Lin, Jianguo; Ooi, Yee-Ming

doi:10.3390/app112210659

Cited by 16 publications

(11 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They utilize dynamic simulators with the capacity to accurately and efficiently simulate the behavior of vehicles to conduct experiments and algorithms. The achievement is exactly the aim that we look forward to, i.e., the autonomous vehicle [ 5 , 6 , 7 , 8 ]. More specifically, in [ 5 ], the authors utilize the transformation in diverse color spaces to design their reward mechanism of Deep Reinforcement Learning (DRL) algorithms.…”

Section: Introductionmentioning

confidence: 90%

“…The achievement is exactly the aim that we look forward to, i.e., the autonomous vehicle [ 5 , 6 , 7 , 8 ]. More specifically, in [ 5 ], the authors utilize the transformation in diverse color spaces to design their reward mechanism of Deep Reinforcement Learning (DRL) algorithms. Since the Hue-Saturation-Value (HSV) model is more closely aligned with the color-making attributes of human vision than the Red-Green-Blue (RGB) model, the former is better for color gradations found in nature [ 9 , 10 ].…”

Section: Introductionmentioning

confidence: 90%

“…Our autonomous driving control strategies are elaborated on in this section. Further, as the description in [ 5 , 7 ], all existing DRL notions proposed in the literature are just frameworks, thus designing a reward mechanism and experimenting with such a mechanism to realize a particular application is still needed. This is the primary contribution of our work.…”

Section: The Drl Control Strategiesmentioning

confidence: 99%

See 2 more Smart Citations

Autonomous Driving Control Based on the Technique of Semantic Segmentation

Tsai

Chang

2023

Sensors

Self Cite

View full text Add to dashboard Cite

Advanced Driver Assistance Systems (ADAS) are only applied to relatively simple scenarios, such as highways. If there is an emergency while driving, the driver should take control of the car to deal properly with the situation at any time. Obviously, this incurs the uncertainty of safety. Recently, in the literature, several studies have been proposed for the above-mentioned issue via Artificial Intelligence (AI). The achievement is exactly the aim that we look forward to, i.e., the autonomous vehicle. In this paper, we realize the autonomous driving control via Deep Reinforcement Learning (DRL) based on the CARLA (Car Learning to Act) simulator. Specifically, we use the ordinary Red-Green-Blue (RGB) camera and semantic segmentation camera to observe the view in front of the vehicle while driving. Then, the captured information is utilized as the input for different DRL models so as to evaluate the performance, where the DRL models include DDPG (Deep Deterministic Policy Gradient) and RDPG (Recurrent Deterministic Policy Gradient). Moreover, we also design an appropriate reward mechanism for these DRL models to realize efficient autonomous driving control. According to the results, only the RDPG strategies can finish the driving mission with the scenario that does not appear/include in the training scenario, and with the help of the semantic segmentation camera, the RDPG control strategy can further improve its efficiency.

show abstract

Section: Introductionmentioning

confidence: 90%

Section: Introductionmentioning

confidence: 90%

Section: The Drl Control Strategiesmentioning

confidence: 99%

See 1 more Smart Citation

Autonomous Driving Control Based on the Technique of Semantic Segmentation

Tsai

Chang

2023

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…Sensors play significant roles in several modern applications [1][2][3][4][5]. Diverse sensors help us obtain various information for different applications.…”

Section: Introductionmentioning

confidence: 99%

Utilizing Ensemble Learning to Improve the Distance Information for UWB Positioning

et al. 2022

Self Cite

View full text Add to dashboard Cite

An ultra-wideband (UWB) positioning system consists of at least three anchors and a tag for the positioning procedure. Via the UWB transceivers mounted on all devices in the system, we can obtain the distance information between each pair of devices and further realize the tag localization. However, the uncertain measurement in the real world may introduce incorrect measurement information, e.g., time, distance, positioning, and so on. Therefore, we intend to incorporate the technique of ensemble learning with UWB positioning to improve its performance. In this paper, we present two methods. The experimental results show that our ideas can be applied to different scenarios and work well. Of note, compared with the existing research in the literature, our first algorithm was more accurate and stable. Further, our second algorithm possessed even better performance than the first. Moreover, we also provide a comprehensive discussion for an ill-advised point, which is often used to evaluate the positioning efficiency in the literature.

show abstract

“…They are usually combined with reinforcement learning (RL) and have achieved great success [6][7][8][9]. Many advanced deep RL algorithms have been proposed and used to solve complex decision-making problems such as game playing [9,10], robotics [11][12][13][14], and autonomous driving [15][16][17][18]. In multi-agent systems [19,20], if an opponent's policy is fixed, the agent can treat it as part of the stationary environment and learn the response policy using single-agent RL algorithms [21][22][23].…”

Section: Introductionmentioning

confidence: 99%

Efficiently Detecting Non-Stationary Opponents: A Bayesian Policy Reuse Approach under Partial Observability

Wang

Chen

et al. 2022

Applied Sciences

View full text Add to dashboard Cite

In multi-agent domains, dealing with non-stationary opponents that change behaviors (policies) consistently over time is still a challenging problem, where an agent usually requires the ability to detect the opponent’s policy accurately and adopt the optimal response policy accordingly. Previous works commonly assume that the opponent’s observations and actions during online interactions are known, which can significantly limit their applications, especially in partially observable environments. This paper focuses on efficient policy detecting and reusing techniques against non-stationary opponents without their local information. We propose an algorithm called Bayesian policy reuse with LocAl oBservations (Bayes-Lab) by incorporating variational autoencoders (VAE) into the Bayesian policy reuse (BPR) framework. Following the centralized training with decentralized execution (CTDE) paradigm, we train VAE as an opponent model during the offline phase to extract the latent relationship between the agent’s local observations and the opponent’s local observations. During online execution, the trained opponent models are used to reconstruct the opponent’s local observations, which can be combined with episodic rewards to update the belief about the opponent’s policy. Finally, the agent reuses the best response policy based on the updated belief to improve online performance. We demonstrate that Bayes-Lab outperforms existing state-of-the-art methods in terms of detection accuracy, accumulative rewards, and episodic rewards in a predator–prey scenario. In this experimental environment, Bayes-Lab can achieve about 80% detection accuracy and the highest accumulative rewards, and its performance is less affected by the opponent policy switching interval. When the switching interval is less than 10, its detection accuracy is at least 10% higher than other algorithms.

show abstract

Autonomous Driving Control Using the DDPG and RDPG Algorithms

Cited by 16 publications

References 11 publications

Autonomous Driving Control Based on the Technique of Semantic Segmentation

Autonomous Driving Control Based on the Technique of Semantic Segmentation

Utilizing Ensemble Learning to Improve the Distance Information for UWB Positioning

Efficiently Detecting Non-Stationary Opponents: A Bayesian Policy Reuse Approach under Partial Observability

Contact Info

Product

Resources

About