Feature Correlation Hypergraph: Exploiting High-order Potentials for Multimodal Recognition

Abstract

In computer vision and multimedia analysis, it is common to use multiple features (or multimodal features) to represent an object. For example, to well characterize a natural scene image, we typically extract a set of visual features to represent its color, texture, and shape. However, it is challenging to integrate multimodal features optimally. Since they are usually high-order correlated, e.g., the histogram of gradient (HOG), bag of scale invariant feature transform descriptors, and wavelets are closely related because they collaboratively reflect the image texture. Nevertheless, the existing algorithms fail to capture the high-order correlation among multimodal features. To solve this problem, we present a new multimodal feature integration framework. Particularly, we first define a new measure to capture the high-order correlation among the multimodal features, which can be deemed as a direct extension of the previous binary correlation. Therefore, we construct a feature correlation hypergraph (FCH) to model the high-order relations among multimodal features. Finally, a clustering algorithm is performed on FCH to group the original multimodal features into a set of partitions. Moreover, a multiclass boosting strategy is developed to obtain a strong classifier by combining the weak classifiers learned from each partition. The experimental results on seven popular datasets show the effectiveness of our approach.

An efficient image aesthetic analysis system using Hadoop

“…Second, we will enrich esthetic analysis model by using various information more than image. For example, multimedia information from the web [31][32][33][34][35][36], or geo-context information of mobile terminals. Third, we will study on designing an adaptive scalable system in which the size of cluster could be automatically adjusted by the amount of requests.…”

Fusing $${\mathcal {R}}$$ R Features and Local Features with Context-Aware Kernels for Action Recognition

“…For example, Liang et al [21] introduce a content-sensitive hypergraph to represent the image and then utilize an incremental hypergraph partitioning to generate candidate regions for the final salient object detection. Zhang et al [58] construct a feature correlation hypergraph to model high order relations among multimodal features for object recognition in images. Li et al [20] construct three types of hypergraphs: the pairwise hypergraph, the k nearest neighbor (kNN) hypergraph, and the high order over-clustering hypergraph.…”

Towards perceptual video cropping with curve fitting

“…Yang et al [15,17] leverage the images tags to tagging or index videos. Therefore we explore the current image cropping method [23], the image first segmented [16,21,22,24] into image regions with semantic tags [13,14] or class label [19,20], then combined with image global configurations and region visual features get a good cropping result which maintains the photo aesthetics. But the time consumption is not acceptable when cropping the video.…”