Close human-robot cooperation is a key enabler for new developments in advanced manufacturing and assistive applications. Close cooperation require robots that can predict human actions and intent, understanding human non-verbal cues. Recent approaches based on neural networks have led to encouraging results in the human action prediction problem both in continuous and discrete spaces. Our approach extends the research in this direction.Our contributions are three-fold. First, we validate the use of gaze and body pose cues as a means of predicting human action through a feature selection method. Next, we address two shortcomings of existing literature: predicting multiple and variable-length action sequences. This is achieved by applying an encoder-decoder recurrent neural network topology in the discrete action prediction problem.In addition, we theoretically demonstrate the importance of predicting multiple action sequences as a means of estimating the stochastic reward in a human robot cooperation scenario.Finally, we show the ability to effectively train the prediction model on an action prediction dataset, involving human motion data, and explore the influence of the model's parameters on its performance.

No abstract

No abstract

Socially assistive robots may help the treatment of autism spectrum disorder(ASD), through games using dyadic interactions to train social skills. Existing systems are mainly based on simplified protocols which qualitatively evaluate subject performance.We propose a robotic coaching platform for training social, motor and cognitive capabilities, with two main contributions: (i) using triadic interactions(adult, robot and child), with robotic mirroring, and (ii) providing quantitative performance indicators. The key system features were accurately designed, including type of protocols, feedback systems and evaluation metrics, contemplating the requirements for applications with ASD children.We implemented two protocols, Robot-Master and Adult-Master, where children performed different gestures guided by the robot or the adult respectively, eventually receiving feedback about movement execution. In both, the robot mirrors the subject during the movement. To assess system functionalities, with a homogeneous group of subjects, tests were carried out with 28 healthy subjects; one preliminary acquisition was done with an ASD child. Data analysis was customized to design protocolspecific parameters for movement characterization.Our tests show that robotic mirroring execution depends on the complexity and standardization of movements, as well as on the robot technical features. The feedback system evaluated movement phases and successfully estimated the completion of the exercises. Future work includes improving platform flexibility and adaptability, and clinical trials with ASD children to test the impact of the robotic coach on reducing symptoms. We trust that the proposed quantitative performance indicators extend the current state-of-the-art towards clinical usage of robotic-based coaching systems.

We introduce TIDEE, an embodied agent that tidies up a disordered scene based on learned commonsense object placement and room arrangement priors. TIDEE explores a home environment, detects objects that are out of their natural place, infers plausible object contexts for them, localizes such contexts in the current scene, and repositions the objects. Commonsense priors are encoded in three modules: i) visuo-semantic detectors that detect out-of-place objects, ii) an associative neural graph memory of objects and spatial relations that proposes plausible semantic receptacles and surfaces for object repositions, and iii) a visual search network that guides the agent's exploration for efficiently localizing the receptacle-of-interest in the current scene to reposition the object. We test TIDEE on tidying up disorganized scenes in the AI2THOR simulation environment. TIDEE carries out the task directly from pixel and raw depth input without ever having observed the same room beforehand, relying only on priors learned from a separate set of training houses. Human evaluations on the resulting room reorganizations show TIDEE outperforms ablative versions of the model that do not use one or more of the commonsense priors. On a related room rearrangement benchmark that allows the agent to view the goal state prior to rearrangement, a simplified version of our model significantly outperforms a top-performing method by a large margin. Code and data are available at the project website: https://tidee-agent.github.io/.