Deriving a discriminative color model for a given object class from weakly labeled training data

Abstract

This paper presents a method for creating a discriminative color model for a given object class based on color occurrence statistics. A discriminative color model can be used to classify individual pixels of images with regards to whether they may belong to the wanted object. However, in contrast to existing approaches, we do not exploit pixel-wise object annotations but only global negative and positive image labels. Therefore our approach requires significantly less manual effort. We quantitatively evaluate the performance of our approach on two publicly available datasets and compare it to a baseline approach, which utilizes pixel annotations. The experimental results show that our approach is on par with pixel-wise approaches although requiring only a single global image label.

Towards automatic object annotations from global image labels

“…The color model itself is a color histogram which corresponds to the model suggested by Jones and Rehg [6]. However, since Jones and Rehg construct their histogram from pixel-wise annotations which we do not have, we follow the approach by Ries and Lienhart [9] which only requires weakly labeled data (i.e. binary image labels).…”

“…We chose the latter variant (with 32 3 bins in YCbCr color space), since color histograms provide a detailed partitioning of the color space at the expense of requiring more storage space. In [9], Ries and Lienhart describe a method for creating a color histogrambased model from global image labels only. Thus, we do not have to violate our assumption about not having manual annotations of regions of interest.…”

“…The main idea of [9] is to statistically determine which colors appear regularly in positive images and less regularly in negative images. Let fN (c) and fP (c) be the relative numbers of negative images and positive images in which color c occurs.…”

“…As suggested in [9] , we use the 0.97 quantile for θc, i.e. we adaptively select the top colors for each class.…”

“…Note that for simplicity, we re-wrote the respective decision function h from [9] by aggregating all constants into θc and by making it a function on a pixel pj.…”