Discovering visual attributes from image and video data

Abstract

Visual attributes have recently attracted great attention from the computer vision community.Their advantages include being: (1) shared across classes, (2) human understandable, and (3) machine computable. Despite these positives, the attribute features in current works are not discriminative enough to be considered as strong features for traditional classification tasks or novel applications. This problem hinders their broad usage. There are several works that primarily focus on increasing the discriminative power of visual attributes. Unfortunately, in most cases, the feature descriptors become hard to understand (i.e., not meaningful); thus, reducing these attribute features into being merely traditional low-level image/video feature descriptors. Furthermore, the discriminative power property and meaningful property are not independent. Some discriminative attributes could be meaningful and vice-versa. For example, the attribute "has hooves" is discriminative for distinguishing between dogs and sheep but the attribute "has four legs" is not. On the other hand, there is significant attention being given to the development of automatic attribute discovery approaches. These approaches focus on automatically discovering potentially meaningful attributes from data without the need for manual labelling. Although it is suggested that the discovered attributes are quite discriminative and meaningful, it is not entirely clear if they are truly meaningful. Quantitative and automatic evaluation methods to determine attribute meaningfulness are desirable in this case, since manual examination is both tedious and time-consuming. Research on this topic can start to shed light on how to automatically and effectively discover meaningful visual attributes without the huge cost of manual labelling.The following research directions have not previously been extensively explored: (1) current works fail to address the problem of finding discriminative and meaningful attributes without involving human effort, and (2) there is no existing way to measure the meaning of discriminative attributes without involving human labelling effort. To that end, this research aims to devise visual attributebased methods for traditional classification tasks and novel applications, including the attribute meaningfulness measurement, by dealing with these two shortcomings. In particular, this thesis addresses the following aspects: (1) automatic discovery of discriminative attributes from a set of meaningful attributes applied to zero-shot learning problems; (2) discovery of meaningful attributes by exploring ways to automatically quantify attribute meaningfulness; the proposed techniques are applied and tested on video keyword generation for video surveillance data, and (3) discovering meaningful and discriminative attributes in fully unsupervised scenarios via multi-graph clustering techniques. The main contribution of this thesis lies in the proposition of the method quantitatively evaluating the meaningfulness of automatic discovered attr...