Hybrid Environment Reconstruction Improving User Experience and Workload in Augmented Virtuality Teleoperation

Blank, Andreas; Kosar, Emir; Karlidag, Engin; Guo, Qiang; Kohn, Sebastian; Sommer, Oliver; Walter, Jonas; Herbert, Manfred; Seßner, Julian; Querfurth, Frank; Metzner, Maximilian; Franke, Jörg

doi:10.1016/j.promfg.2021.10.007

Cited by 2 publications

(3 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hybrid learning combines known RL and IL algorithms by formulating weighted objective functions within shared constraints. In computer science, AV refers to the augmentation of the Virtual Reality (VR) with realworld elements, enriching the user experience [5]. The overall objective of the method is to reduce required adaptation efforts to new and changing scenarios.…”

Section: Motivationmentioning

confidence: 99%

“…In AV mode, however, the real-world robot environment is rendered by a soft real-time capable environment reconstruction pipeline. The latter operates with multiple sensor data inputs and their subsequent processing and characterization (e. g. object localization, pose estimation and knowledge augmentation) [5]. The IL stack (Fig.…”

Section: Augmented Virtuality-based Hybrid Manipulation Learningmentioning

confidence: 99%

“…Unity ML-Agents is utilized as the Deep Learning API for episode design and runtime environment for demonstrations, training and inference. An HTC Vive Pro serves as HMI, thereby the SteamVR Unity plugin and OpenVR are used [5]. On a second IPC, the Robot Operating System (ROS) is installed for communication with the robot.…”

Section: Demonstrator Setupmentioning

confidence: 99%

See 2 more Smart Citations

Augmented Virtuality Input Demonstration Refinement Improving Hybrid Manipulation Learning for Bin Picking

Blank

Zikeli

Reitelshöfer

et al. 2022

Lecture Notes in Mechanical Engineering

Self Cite

View full text Add to dashboard Cite

Beyond conventional automated tasks, autonomous robot capabilities aside human cognitive skills are gaining importance in industrial applications. Although machine learning is a major enabler of autonomous robots, system adaptation remains challenging and time-consuming. The objective of this research work is to propose and evaluate an augmented virtuality-based input demonstration refinement method improving hybrid manipulation learning for industrial bin picking. To this end, deep reinforcement and imitation learning are combined to shorten required adaptation timespans to new components and changing scenarios. The method covers initial learning and dataset tuning during ramp-up as well as fault intervention and dataset refinement. For evaluation standard industrial components and systems serve within a real-world experimental bin picking setup utilizing an articulated robot. As part of the quantitative evaluation, the method is benchmarked against conventional learning methods. As a result, required annotation efforts for successful object grasping are reduced. Thereby, final grasping success rates are increased. Implementation samples are available on: https://github.com/FAU-FAPS/hybrid_manipulationlearning_unity3dros

show abstract

Section: Motivationmentioning

confidence: 99%

Section: Augmented Virtuality-based Hybrid Manipulation Learningmentioning

confidence: 99%