End-to-End Automatic Image Annotation Based on Deep CNN and Multi-Label Data Augmentation

Xiao, Ke; Zou, Jiawei; Niu, Yuzhen

doi:10.1109/tmm.2019.2895511

Cited by 88 publications

(40 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many studies have been conducted on using photo contents to extract travel location information, which is based on color, texture, or shape representation [25][26][27]. Ke et al [25] proposed a method that transforms the photo annotation problem into a multi-label learning problem. Kuang et al [26] estimated visual information from photos by associated tags in a local region.…”

Section: Using Extra Data To Address the Travel Location Cold Start Pmentioning

confidence: 99%

A Convolutional Neural Network and Matrix Factorization-Based Travel Location Recommendation Method Using Community-Contributed Geotagged Photos

Ameen

Chen

et al. 2020

IJGI

View full text Add to dashboard Cite

Travel location recommendation methods using community-contributed geotagged photos are based on past check-ins. Therefore, these methods cannot effectively work for new travel locations, i.e., they suffer from the travel location cold start problem. In this study, we propose a convolutional neural network and matrix factorization-based travel location recommendation method to address the problem. Specifically, a weighted matrix factorization method is used to obtain the latent factor representations of travel locations. The latent factor representation for a new travel location is estimated from its photos by using a convolutional neural network. Experimental results on a Flickr dataset demonstrate that the proposed method can provide better recommendations than existing methods.

show abstract

Section: Using Extra Data To Address the Travel Location Cold Start Pmentioning

confidence: 99%

A Convolutional Neural Network and Matrix Factorization-Based Travel Location Recommendation Method Using Community-Contributed Geotagged Photos

Ameen

Chen

et al. 2020

IJGI

View full text Add to dashboard Cite

show abstract

“…Based on initial labels extracted from the CNNs and initial labels of possibly user-defined tags, the event categories and final annotations of the images were estimated through a refinement process based on the expectation-maximization (EM) algorithm. Ke et al [ 19 ] developed an end-to-end automatic image annotation model based on a deep convolutional neural network (E2E-DCNN) and multilabel data augmentation. A deep CNN structure was adopted for adaptive feature learning, in which the cross-entropy loss functions were first used to construct an end-to-end annotation structure for training, and Wasserstein generative adversarial networks were used for multilabel data augmentation.…”

Section: Related Workmentioning

confidence: 99%

Automatic image annotation method based on a convolutional neural network with threshold optimization

2020

View full text Add to dashboard Cite

In this study, a convolutional neural network with threshold optimization (CNN-THOP) is proposed to solve the issue of overlabeling or downlabeling arising during the multilabel image annotation process in the use of a ranking function for label annotation along with prediction probability. This model fuses the threshold optimization algorithm to the CNN structure. First, an optimal model trained by the CNN is used to predict the test set images, and batch normalization (BN) is added to the CNN structure to effectively accelerate the convergence speed and obtain a group of prediction probabilities. Second, threshold optimization is performed on the obtained prediction probability to derive an optimal threshold for each class of labels to form a group of optimal thresholds. When the prediction probability for this class of labels is greater than or equal to the corresponding optimal threshold, this class of labels is used as the annotation result for the image. During the annotation process, the multilabel annotation for the image to be annotated is realized by loading the optimal model and the optimal threshold. Verification experiments are performed on the MIML, COREL5K, and MSRC datasets. Compared with the MBRM, the CNN-THOP increases the average precision on MIML, COREL5K, and MSRC by 27%, 28% and 33%, respectively. Compared with the E2E-DCNN, the CNN-THOP increases the average recall rate by 3% on both COREL5K and MSRC. The most precise annotation effect for CNN-THOP is observed on the MIML dataset, with a complete matching degree reaching 64.8%.

show abstract

“…Three benchmark dataset which are used widely are selected for evaluating the proposed work. Corel5K [43], Espgame [47], and Iaprtc12 [45]. Corel5K contains 5000 images which classify 4000 images as train image and 1000 images as test images.…”

Section: A Datasetsmentioning

confidence: 99%

CSL Net: Convoluted SE and LSTM Blocks Based Network for Automatic Image Annotation

2019

IJEAT

View full text Add to dashboard Cite

Due to advancement of multimedia technology, availability and usage of image and video data is enormous. For indexing and retrieving those data, there is a need for an efficient technique. Now, Automatic keyword generation for images is a focussed research which has lot of attractions. In general, conventional auto annotation methods having lesser performance over deep learning methods. The annotation is transformed as captioning in deep learning models. In this paper, we propose a new model CSL Net (CSLN) as a combination of convoluted squeeze and excitation block with Bi-LSTM blocks to predict tags for images. The proposed model is evaluated using the various benchmark datasets like CIFAR10, Corel5K, ESPGame and IAPRTC12. It is observed that, the proposed work yields better results compared to that of the existing methods in term of precision, recall and accuracy.

show abstract

End-to-End Automatic Image Annotation Based on Deep CNN and Multi-Label Data Augmentation

Cited by 88 publications

References 38 publications

A Convolutional Neural Network and Matrix Factorization-Based Travel Location Recommendation Method Using Community-Contributed Geotagged Photos

A Convolutional Neural Network and Matrix Factorization-Based Travel Location Recommendation Method Using Community-Contributed Geotagged Photos

Automatic image annotation method based on a convolutional neural network with threshold optimization

CSL Net: Convoluted SE and LSTM Blocks Based Network for Automatic Image Annotation

Contact Info

Product

Resources

About