Human-Centered Collaborative Robots With Deep Reinforcement Learning

Ghadirzadeh, Ali; Chen, Xi; Yin, Wenjie; Yi, Zhengrong; Björkman, Mårten; Kragić, Danica

doi:10.1109/lra.2020.3047730

Cited by 62 publications

(34 citation statements)

References 29 publications

(39 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In deep reinforcement learning, the single-step average reward value of each episode is an important indicator to measure the training effect [ 30 , 31 , 32 , 33 ]. This paper counts the average single-step rewards of [ 22 ] and DCPER-DDPG algorithm in 6000 episodes.…”

Section: Results Analysismentioning

confidence: 99%

Lane Following Method Based on Improved DDPG Algorithm

Zhang

et al. 2021

Sensors

View full text Add to dashboard Cite

In an autonomous vehicle, the lane following algorithm is an important component, which is a basic function of autonomous driving. However, the existing lane following system has a few shortcomings: first, the control method it adopts requires an accurate system model, and different vehicles have different parameters, which needs a lot of parameter calibration work. The second is that it may fail on road sections where the lateral acceleration requirements of vehicles are large, such as large curves. Third, its decision-making system is defined based on rules, which has disadvantages: it is difficult to formulate; human subjective factors cannot guarantee objectivity; coverage is difficult to guarantee. In recent years, the deep deterministic policy gradient (DDPG) algorithm has been widely used in the field of autonomous driving due to its strong nonlinear fitting ability and generalization performance. However, the DDPG algorithm has overestimated state action values and large cumulative errors, low training efficiency and other issues. Therefore, this paper improves the DDPG algorithm based on the double critic networks and priority experience replay mechanism. Then this paper proposes a lane following method based on this algorithm. Experiment shows that the algorithm can achieve excellent following results under various road conditions.

show abstract

Section: Results Analysismentioning

confidence: 99%

Lane Following Method Based on Improved DDPG Algorithm

Zhang

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…(1) Creation of 3D virtual models from the experimental assembly by any 3D design software or point cloud creation by laser scanning technology with conversion to some standard 3D format (OBJ, FBX, STL, IGES, etc. ); (2) Import 3D models into the software with cinematic rendering and some simulation of dynamics; (3) Algorithms design of an automatic data queue of parts positioning, rotating, and camera setup by parts size; (4) Rendering two sets of images: the first for CNN teaching and the second for an automated annotation algorithm; (5) Creating of XML file for single shot detection and JSON format for instance segmentation; (6) Automated ratio sorting to training and testing samples and moving to separate folder; (7) Training of convolutional neural network for parts classification and localization (using single shot detection and instance segmentation); (8) Transformation of CNN models into some type of embedded devices for inference of the trained model and results distribution of the detected position data to assisted assembly systems: a collaborative robot internal Cartesian system and mixed reality device anchoring system;…”

Section: Methodology Of Deep Learning Implementation Into the Assisted Assembly Processmentioning

confidence: 99%

“…A nice review of virtual, mixed, and augmented reality for immersive systems research is presented in [4]. Some other research results of the mixed assembly process between human and collaborative robots are described in [5][6][7]. An AR-based worker support system for human-robot collaboration using AR libraries was proposed in [8] and an anchoring support system using the AR toolkit was developed in [9].…”

Section: Introduction and Related Workmentioning

confidence: 99%

CNN Training Using 3D Virtual Models for Assisted Assembly with Mixed Reality and Collaborative Robots

et al. 2021

View full text Add to dashboard Cite

The assisted assembly of customized products supported by collaborative robots combined with mixed reality devices is the current trend in the Industry 4.0 concept. This article introduces an experimental work cell with the implementation of the assisted assembly process for customized cam switches as a case study. The research is aimed to design a methodology for this complex task with full digitalization and transformation data to digital twin models from all vision systems. Recognition of position and orientation of assembled parts during manual assembly are marked and checked by convolutional neural network (CNN) model. Training of CNN was based on a new approach using virtual training samples with single shot detection and instance segmentation. The trained CNN model was transferred to an embedded artificial processing unit with a high-resolution camera sensor. The embedded device redistributes data with parts detected position and orientation into mixed reality devices and collaborative robot. This approach to assisted assembly using mixed reality, collaborative robot, vision systems, and CNN models can significantly decrease assembly and training time in real production.

show abstract

“…On the one hand, Al-Yacoub et al present in [140] a LfD methodology that combines a machine learning algorithm -i.e., Random Forest (RF)-with stochastic regression, using haptic information captured from human demonstration. On the other hand, Ghadirzadeh et al propose in [141] a RL based framework for a more time-efficient HR cooperation that finds an optimal balance between timely actions and the risk of taking improper actions.…”

Section: Efficiency-oriented Control System Designmentioning

confidence: 99%

“…All the aforementioned control strategies based on learning have been developed to address uncertainties and external disturbances that might provoke the robot's performance degradation by replacing the traditional proportional-integralderivative controllers, typically characterized by a complicated tuning of control parameters. Since the above described methods suffer from several problems, ranging from a huge computation time to a limited generalizability or adaptability to unseen situations, NNs based on modern control theories -e.g., SMC [89], [90], Takagi-Sugeno fuzzy control [57], and RL [70], [109], [112], [121], [133], [141]-are introduced in the literature and modeled to overcome these complex robot's control issues. Obviously, also these innovative advanced techniques present limitations, such as chattering and sensitive problems for the SMC and possible instabilities for fuzzy approaches.…”

Section: B Emerging Control Issues and Challengesmentioning

confidence: 99%

Control Techniques for Safe, Ergonomic, and Efficient Human-Robot Collaboration in the Digital Industry: A Survey

Proia

Carli

Cavone

et al. 2022

IEEE Trans. Automat. Sci. Eng.

View full text Add to dashboard Cite

The fourth industrial revolution, also known as Industry 4.0, is reshaping the way individuals live and work while providing a substantial influence on the manufacturing scenario.The key enabling technology that has made Industry 4.0 a concrete reality is without doubt collaborative robotics, which is also evolving as a fundamental pillar of the next revolution, the so-called Industry 5.0. The improvement of employees' safety and well-being, together with the increase of profitability and productivity, are indeed the main goals of human-robot collaboration (HRC) in the industrial setting. The robotic controller design and the analysis of existing decision and control techniques are crucially needed to develop innovative models and state-of-the-art methodologies for a safe, ergonomic, and efficient HRC. To this aim, this paper presents an accurate review of the most recent and relevant contributions to the related literature, focusing on the control perspective. All the surveyed works are carefully selected and categorized by target (i.e., safety, ergonomics, and efficiency), and then by problem and type of control, in presence or absence of optimization. Finally, the discussion of the achieved results and the analysis of the emerging challenges in this research field are reported, highlighting the identified gaps and the promising future developments in the context of the digital evolution.Note to Practitioners-The design and development of manufacturing systems are experiencing substantial changes towards full automation. This ongoing challenge is being tackled by academia and industrial practitioners with the adoption of collaborative robots, where the skills and peculiarities of humans (e.g., intelligence, creativity, adaptability, etc.) and robots (e.g., flexibility, pinpoint accuracy, tirelessness, etc.) are combined to better perform a variety of tasks. Nevertheless, due to their different characteristics, there is an emerging need for designing suitable decision and control techniques to ensure a safe and ergonomic HRC, while keeping the highest level of productivity. Against this background, the aim of this paper is to provide researchers and practitioners with a reference source in the related field, which can help them designing and developing suitable solutions to control problems in safe, ergonomic, and efficient collaborative robotics.

show abstract

Human-Centered Collaborative Robots With Deep Reinforcement Learning

Cited by 62 publications

References 29 publications

Lane Following Method Based on Improved DDPG Algorithm

Lane Following Method Based on Improved DDPG Algorithm

CNN Training Using 3D Virtual Models for Assisted Assembly with Mixed Reality and Collaborative Robots

Control Techniques for Safe, Ergonomic, and Efficient Human-Robot Collaboration in the Digital Industry: A Survey

Contact Info

Product

Resources

About