Temporal Human Action Segmentation via Dynamic Clustering

Abstract

We present an effective dynamic clustering algorithm for the task of temporal human action segmentation, which has comprehensive applications such as robotics, motion analysis, and patient monitoring. Our proposed algorithm is unsupervised, fast, generic to process various types of features, and applicable in both the online and offline settings. We perform extensive experiments of processing data streams, and show that our algorithm achieves the state-ofthe-art results for both online and offline settings.

Local Temporal Bilinear Pooling for Fine-Grained Action Parsing

“…Such feature mapping is equivalent to the bilinear form, in the sense that the associated kernel function, and hence the reproducing-kernel Hilbert space (RKHS), is identical. (3) We perform extensive experiments to investigate our novel bilinear pooling methods, and show that the proposed method consistently improves or is on-par with the performance of the state-of-the-art methods on diverse datasets. To our knowledge, we are the first to employ bilinear pooling in a convolutional encoder-decoder architecture for fine-grained action parsing over time.…”

“…Parsing fine-grained actions over time is important in many applications, which require understanding of subtle and precise operations over long-term periods, e.g. daily activities [1], surgical robots [2], human motion analysis [3] and animal behavior analysis in the lab [4]. Given a video or a generic time sequence of feature vectors, an action parsing algorithm aims at assigning each frame an action label, such that the entire sequence is partitioned into several disjoint semantic action primitives.…”

“…(2) The conventional bilinear pooling aggregates the outer products of the feature vectors by averaging, and hence loses representativeness when the real data distribution is complex. (3) The conventional bilinear pooling lifts the feature dimension from d to d 2 , causing parameter proliferation in the neural net and expensive computational cost.…”