Cograph Regularized Collective Nonnegative Matrix Factorization for Multilabel Image Annotation

Abstract

Automatic image annotation is an effective and straightforward way to facilitate many applications in computer vision. However, manually annotating images is a computation-expensive and labor-intensive task. To address these problems, this paper proposes a novel approach by using a cograph regularized collective nonnegative matrix factorization method to annotate images, which is referred to as CG-CNMF; CG-CNMF maximizes the annotation consistency for each image and minimizes the semantic gap for good annotation performance. To reduce the computation cost, this method formulates the annotation problem as a recommending issue and uses nonnegative matrix factorization (NMF) to recover the image-tolabel relation for the testing images. Moreover, to find the most similar latent image features and latent label features during the matrix factorization, it exploits the image-to-image relation and label-to-label relation by utilizing the visual content information of images and the semantic cooccurrence information of labels, respectively. To reduce the semantic gap between the image visual content and semantic concepts, both the semantic features and convolutional neural networks (CNNs)-based visual features are considered. Moreover, to address the label-imbalance and incomplete-label problems, the visual-based label cooccurrence information is also considered. In this way, visually similar images are highly correlated with the true semantics of the test images. The experimental results for three multilabel image datasets demonstrate the effectiveness and the efficiency of the proposed method.

Graph Convolutional Networks based Non-Negative Matrix Factorization aware Community detection

“…In addition, community discovery is essentially a node clustering problem, so NMF is also suitable for community discovery. At present, there are many community discovery methods based on NMF, mainly by extending the basic NMF model to solve the community discovery problem of various complex networks, for example, community discovery method SNMF based on symmetric NMF [16,27], community-based on joint NMF The discovery method S2-jNMF [28], the semi-supervised NMF-based community method SPOCD [29], SMpC [30] and the deep NMF-based community method DANMF [31], etc. Although these methods have achieved specific performance improvements to varying degrees, since NMF is essentially a linear model, these methods are still linear and do not have nonlinear expression capabilities.…”

Central Attention with Multi-graphs for Image Annotation

“…Meanwhile, [43] utilizes the multi-view to construct graph structures. This method [46], [48] takes into account the relationship between samples, the relationship between labels and labels, and the relationship between samples and labels. In order to get the utmost out of these three relationships, a set matrix decomposition model is proposed to simultaneously decompose three relationship matrices.…”

“…2) The model with graph structure [13,14,23,25,40] ponder manifold structure information between images, between labels or between images and tags. A multitude of graph structures, such as Locally Linear Embedding(LLE) [25], Laplacian eigenmaps(LE) [23], [43,46,48], Hypergraph [36], [40] and Hessian [11,28], are embedded into the models of AIA. Meanwhile, GCN [39] is also applied to solve the problem of AIA.…”

Adaptive bag‐of‐visual word modelling using stacked‐autoencoder and particle swarm optimisation for the unsupervised categorisation of images

“…Deep Feature Learning has been recognised in image retrieval researches as a reliable method for generating a highlevel image representation from a massive collection of images [16,16,40,41,42,43,44], and has been found to be an important inclusion in the implementation of automatic image annotation due to its strong discriminatory power of Deep Learning Image representations [41,45].…”