Unsupervised, Semi-Supervised Interactive Force Estimations During pHRI via Generated Synthetic Force Myography Signals

Abstract

Recognizing applied hand forces using force myography (FMG) biosignals requires adequate training data to facilitate physical human-robot interactions (pHRI). But in practice, data is often scarce, and labels are usually unavailable or time consuming to generate. Synthesizing FMG biosignals can be a viable solution. Therefore, in this paper, we propose for the first time a dual-phased algorithm based on semisupervised adversarial learning utilizing fewer labeled real FMG data with generated unlabeled synthetic FMG data. We conducted a pilot study to test this algorithm in estimating applied forces during interactions with a Kuka robot in 1D-X, Y, Z directions. Initially, an unsupervised FMG-based deep convolutional generative adversarial network (FMG-DCGAN) model was employed to generate real-like synthetic FMG data. A variety of transformation functions were used to observe domain randomization for increasing data variability and for representing authentic physiological, environmental changes. Cosine similarity score and generated-to-input-data ratio were used as decision criteria minimizing the reality gap between real and synthetic data and helped avoid risks associated with wrong predictions. Finally, the FMG-DCGAN model was pretrained to generate pseudo-labels for unlabeled real and synthetic data, further retrained using all labeled and pseudo-labeled data and was termed as the self-trained FMG-DCGAN model. Lastly, this model was evaluated on unseen real test data and achieved accuracies of 85%>R 2 >77% in force estimation compared to the corresponding supervised baseline model (89%>R 2 >78%). Therefore, the proposed method can be more practical for use in FMG-based HRI, rehabilitation, and prosthetic control for daily, repetitive usage even with few labeled data.

Detecting Safety Anomalies in pHRI Activities via Force Myography

“…In this study, observing human–robot interactions via FMG biosignals provided a method to detect unsafe human activities. In our previous studies [ 48 , 49 , 50 , 51 , 52 , 53 ], human intentions during motions and applied interactive forces for pHRI activities were successfully incorporated in effective control system designs. In addition to the compliant collaboration, this proposed unwanted pHRI activity detection system will provide an extra layer of protection and will improve occupational health and safety monitoring.…”

“…Common industrial tasks such as object transportation or handover may require the worker to apply hand forces during interactions with the robot. Estimating the applied hand force to interact with a robot in dynamic motions using FMG signals was found favorable in physical interactions [ 48 , 49 , 50 , 51 , 52 , 53 ]. These studies implemented compliant collaboration, where a robot’s trajectory was dictated by the applied hand force of a human worker.…”