Anticipation in Human-Robot Cooperation: A Recurrent Neural Network Approach for Multiple Action Sequences Prediction

Schydlo, Paul; Raković, Mirko; Jamone, Lorenzo; Santos-Victor, José

doi:10.1109/icra.2018.8460924

Cited by 62 publications

(56 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…these functions do not need to be differentiable. Similar issues have been known before, for related strategies (albeit employing SHP models regarding our definition) we refer to [16].…”

Section: Sequence Generationmentioning

confidence: 93%

Ambiguity in Sequential Data: Predicting Uncertain Futures With Recurrent Models

Berlati

Scheel

Stefano

et al. 2020

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Ambiguity is inherently present in many machine learning tasks, but especially for sequential models seldom accounted for, as most only output a single prediction. In this work we propose an extension of the Multiple Hypothesis Prediction (MHP) model to handle ambiguous predictions with sequential data, which is of special importance, as often multiple futures are equally likely. Our approach can be applied to the most common recurrent architectures and can be used with any loss function. Additionally, we introduce a novel metric for ambiguous problems, which is better suited to account for uncertainties and coincides with our intuitive understanding of correctness in the presence of multiple labels. We test our method on several experiments and across diverse tasks dealing with time series data, such as trajectory forecasting and maneuver prediction, achieving promising results.

show abstract

“…these functions do not need to be differentiable. Similar issues have been known before, for related strategies (albeit employing SHP models regarding our definition) we refer to [16].…”

Section: Sequence Generationmentioning

confidence: 93%

Ambiguity in Sequential Data: Predicting Uncertain Futures With Recurrent Models

Berlati

Scheel

Stefano

et al. 2020

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

show abstract

“…This factor was obtained by 3-fold cross-validation, and the average is reported. F1 factor is defined as Schydlo et al (2018)…”

Section: Sample Collaboration Scenariosmentioning

confidence: 99%

“…Probabilistic models, e.g., machine learning algorithms that do not require a complete model of human behavior, have been employed to process the data collected by sensors. Hidden Markov model and neural networks are examples of methods used for estimating human intentions in collaboration with a robot (Wang et al, 2009 ; Ge et al, 2011 ; Ravichandar and Dani, 2017 ; Schydlo et al, 2018 ).…”

Section: Introductionmentioning

confidence: 99%

FMG- and RNN-Based Estimation of Motor Intention of Upper-Limb Motion in Human-Robot Collaboration

et al. 2020

View full text Add to dashboard Cite

Research on human-robot interactions has been driven by the increasing employment of robotic manipulators in manufacturing and production. Toward developing more effective human-robot collaboration during shared tasks, this paper proposes an interaction scheme by employing machine learning algorithms to interpret biosignals acquired from the human user and accordingly planning the robot reaction. More specifically, a force myography (FMG) band was wrapped around the user's forearm and was used to collect information about muscle contractions during a set of collaborative tasks between the user and an industrial robot. A recurrent neural network model was trained to estimate the user's hand movement pattern based on the collected FMG data to determine whether the performed motion was random or intended as part of the predefined collaborative tasks. Experimental evaluation during two practical collaboration scenarios demonstrated that the trained model could successfully estimate the category of hand motion, i.e., intended or random, such that the robot either assisted with performing the task or changed its course of action to avoid collision. Furthermore, proximity sensors were mounted on the robotic arm to investigate if monitoring the distance between the user and the robot had an effect on the outcome of the collaborative effort. While further investigation is required to rigorously establish the safety of the human worker, this study demonstrates the potential of FMG-based wearable technologies to enhance human-robot collaboration in industrial settings.

show abstract

“…Based on 3D skeleton sequences, Liu et al [17] proposed a scale selection network composed of a dilated convolutional network and an activation sharing scheme. Schydlo et al [18] applied an encoder-decoder recurrent neural network for predicting multiple and variable-length sequences, and introduced simultaneous prediction for this problem.…”

Section: Related Workmentioning

confidence: 99%

A Multi-Task Framework for Action Prediction

Liu

Yan

et al. 2020

Information

View full text Add to dashboard Cite

Predicting the categories of actions in partially observed videos is a challenging task in the computer vision field. The temporal progress of an ongoing action is of great importance for action prediction, since actions can present different characteristics at different temporal stages. To this end, we propose a novel multi-task deep forest framework, which treats temporal progress analysis as a relevant task to action prediction and takes advantage of observation ratio labels of incomplete videos during training. The proposed multi-task deep forest is a cascade structure of random forests and multi-task random forests. Unlike the traditional single-task random forests, multi-task random forests are built upon incomplete training videos annotated with action labels as well as temporal progress labels. Meanwhile, incorporating both random forests and multi-task random forests can increase the diversity of classifiers and improve the discriminative power of the multi-task deep forest. Experiments on the UT-Interaction and the BIT-Interaction datasets demonstrate the effectiveness of the proposed multi-task deep forest.

show abstract

Anticipation in Human-Robot Cooperation: A Recurrent Neural Network Approach for Multiple Action Sequences Prediction

Cited by 62 publications

References 24 publications

Ambiguity in Sequential Data: Predicting Uncertain Futures With Recurrent Models

Ambiguity in Sequential Data: Predicting Uncertain Futures With Recurrent Models

FMG- and RNN-Based Estimation of Motor Intention of Upper-Limb Motion in Human-Robot Collaboration

A Multi-Task Framework for Action Prediction

Contact Info

Product

Resources

About