Learning Graphs to Model Visual Objects across Different Depictive Styles

Abstract

Abstract. Visual object classification and detection are major problems in contemporary computer vision. State-of-art algorithms allow thousands of visual objects to be learned and recognized, under a wide range of variations including lighting changes, occlusion, point of view and different object instances. Only a small fraction of the literature addresses the problem of variation in depictive styles (photographs, drawings, paintings etc.). This is a challenging gap but the ability to process images of all depictive styles and not just photographs has potential value across many applications. In this paper we model visual classes using a graph with multiple labels on each node; weights on arcs and nodes indicate relative importance (salience) to the object description. Visual class models can be learned from examples from a database that contains photographs, drawings, paintings etc. Experiments show that our representation is able to improve upon Deformable Part Models for detection and Bag of Words models for classification.

The Art of Detection

“…Our previous work [13] demonstrates that this patch-based method can be extended to object categories in paintings beyond the instance matching of [5]. Others [41,42] have considered the wider problem of generalising across many depictive styles (e.g. photo, cartoon, painting) by building a depiction-invariant graph model.…”

Cross-depiction problem: Recognition and synthesis of photographs and artwork

“…A less computationally intensive approach has been proposed [34] using a hierarchical graph model to obtain a coarse-to-fine arrangement of parts with nodes labelled by qualitative shape [35]. Wu et al [36] address the cross-depiction problem using a deformable model; they use a fully connected graph with learned weights on nodes (the importance of nodes to discriminative classification), on edges (by analogy, the stiffness of a spring connecting parts), and multiple node labels (to account to different depictions); a method tested on 50 categories. Others use no labels at all, but rely on connection structure alone [37] or distances between low-level parts [17].…”

“…We evaluate the algorithms on Photo-Art-50 dataset [36] which contains 50 distinct object classes (see Fig. 2), with between 90 and 138 images for each class.…”

“…The fully connected multi-labelled graph (MG) model [36] is designed for the cross-depiction problem. It attempts to separate appearance features (contingent on the details of a particular depiction) from the information that characterises an object class without reference to any depiction.…”

“…DPM [55] used a strong spatial layout model, and ADPM is our domain adapted version. Multi-labelled graphs (MG) [36] has a stronger spatial model than DPM, and also has two labels at each node. We have include a deep learning CNN [38] too.…”