Baselines for Image Annotation

Abstract

Abstract. Automatically assigning keywords to images is of great interest as it allows one to index, retrieve, and understand large collections of image data. Many techniques have been proposed for image annotation in the last decade that give reasonable performance on standard datasets. However, most of these works fail to compare their methods with simple baseline techniques to justify the need for complex models and subsequent training. In this work, we introduce a new baseline technique for image annotation that treats annotation as a retrieval problem. The proposed technique utilizes low-level image features and a simple combination of basic distances to find nearest neighbors of a given image. The keywords are then assigned using a greedy label transfer mechanism. The proposed baseline outperforms the current state-of-the-art methods on two standard and one large Web dataset. We believe that such a baseline measure will provide a strong platform to compare and better understand future annotation techniques.

Exploring SVM for Image Annotation in Presence of Confusing Labels

“…We use the same evaluation criteria as being used by previous methods [6,8,11,19,23]. Given a new sample, first we compute the score for each label using the corresponding classifier, and then assign it the five top-scoring labels.…”

“…(b) Visual similarity based score: We compute the visual similarity score sim(·) (scaled into range [0, 1]) of x j with its nearest neighbour x * i ∈ S + using JEC [11] method and define…”

“…We use three datasets popular in the image annotation task [5,8,11,19]. These are Corel5k [4], ESP Game [20] and IAPRTC-12 [7].…”

“…This has applications in several tasks such as image retrieval, object recognition, robot navigation, etc. Hence this has emerged as an important research area during the last decade [2,5,6,8,11,19,23]. In annotation datasets with large vocabularies of few hundred or more labels, there exist three practical issues: (a) Incomplete-labeling: The training samples are not exhaustively tagged with all relevant labels from vocabulary.…”

“…Among the image annotation models being proposed in the past, generative or nearestneighbour (NN)-based models [5,8,11,19,23] have particularly been shown to be successful for large vocabulary datasets such as Corel-5k [4], ESP Game [20] and IAPRTC-12 [7]. The reason behind this is that in NN-based models, given a sample, the labels that are not present in the ground-truth of its neighbouring samples are simply ignored, rather than being considered as negative.…”