Learned features versus engineered features for semantic video indexing

Abstract

In this paper, we compare "traditional" engineered (hand-crafted) features (or descriptors) and learned features for content-based semantic indexing of video documents. Learned (or semantic) features are obtained by training classifiers for other target concepts on other data. These classifiers are then applied to the current collection. The vector of classification scores is the new feature used for training a classifier for the current target concepts on the current collection. If the classifiers used on the other collection are of the Deep Convolutional Neural Network (DCNN) type, it is possible to use as a new feature not only the score values provided by the last layer but also the intermediate values corresponding to the output of all the hidden layers. We made an extensive comparison of the performance of such features with traditional engineered ones as well as with combinations of them. The comparison was made in the context of the TRECVid semantic indexing task. Our results confirm those obtained for still images: features learned from other training data generally outperform engineered features for concept recognition. Additionally, we found that directly training SVM classifiers using these features does significantly better than partially retraining the DCNN for adapting it to the new data. We also found that, even though the learned features performed better that the engineered ones, the fusion of both of them perform significantly better, indicating that engineered features are still useful, at least in this case.

Human action recognition by means of subtensor projections and dense trajectories

“…Even though learned features (CNN) could obtain better results than ours when large datasets are available, engineered features can be still useful when few training samples are available as proved in [57]. Moreover, engineered features like ours are still useful because they can improve the performance when combined with CNNs features, as suggested in [58].…”

“…In recent years, most of authors combine their CNNs techniques with IDTs features to improve the results. As suggested in [58], engineered features are still useful because they can improve the performance when combined with CNNs features. In the future we will test if our features are complementary with CNNs features to improve the state-of-the-art results.…”

Automatic Speech Feature Learning for Continuous Prediction of Customer Satisfaction in Contact Center Phone Calls

“…For instance, in computer vision novel feature learning techniques are applied directly on the raw pixel representations of images avoiding the signal parameterization or any other prior preprocessing [15]. Budnik et al [3] report an extensive comparison of the performance of CNN based features with traditional engineered ones, as well as with combinations of them, in the framework of the TRECVid semantic indexing task.…”