3D skeleton-based motion prediction and activity recognition are two interwoven tasks in human behaviour analysis. In this work, we propose a motion context modeling that provides a new way to combine the advantages of both graph convolutional neural network and recurrent neural network for joint human motion prediction and activity recognition. Our approach is based on an LSTM encoder-decoder and a non-local feature extraction attention mechanism to model the spatial correlation of human skeleton data and temporal correlation among motion frames. The proposed network can easily include two output branches, Activity Recognition and Future Motion Prediction, and make them trained jointly to consolidate each other. Experimental results on Human 3.6M, CMU Mocap and NTU RGB-D dataset show that our proposed approach provides the best prediction capability among baseline LSTMbased methods, while obtaining a comparable performance with other state-of-the-art methods.

No abstract

In this paper, a grounding framework is proposed that combines unsupervised and supervised grounding by extending an unsupervised grounding model with a mechanism to learn from explicit human teaching. To investigate whether explicit teaching improves the sample efficiency of the original model, both models are evaluated through an interaction experiment between a human tutor and a robot in which synonymous shape, color, and action words are grounded through geometric object characteristics, color histograms, and kinematic joint features. The results show that explicit teaching improves the sample efficiency of the unsupervised baseline model.