Action recognition with low observational latency via part movement model

Abstract

In this paper, we address the issue of recognizing human actions with low observational latency, which is vital for many applications such as virtual reality and interactive entertainment. Then our purpose is to achieve competitive action recognition performance from very short video clips. Such a task essentially is challenging because only very limited information is provided. To this end, we first develop a feature extraction method to exploit both motion (local flow) and appearance (local shape) features such that the information insufficiency can be effectively mitigated. Then we propose an action representation method named Part Movement Model (PMM), which explicitly captures the spatial-temporal structure of human actions and divides the actions into discriminative part movements. Consequently, the actions can be better represented and the competitive performance can be achieved although only short clips are used. Finally, we experimentally verify the effectiveness of the proposed method on three benchmark datasets. The results show that short clips of 6 − 7 frames (0.2 − 0.3 second video) are enough to achieve the recognition performance comparable to the baselines with high latency.