Bayesian Joint Topic Modelling for Weakly Supervised Object Localisation

Abstract

We address the problem of localisation of objects as bounding boxes in images with weak labels. This weakly supervised object localisation problem has been tackled in the past using discriminative models where each object class is localised independently from other classes. We propose a novel framework based on Bayesian joint topic modelling. Our framework has three distinctive advantages over previous works: (1) All object classes and image backgrounds are modelled jointly together in a single generative model so that "explaining away" inference can resolve ambiguity and lead to better learning and localisation. (2) The Bayesian formulation of the model enables easy integration of prior knowledge about object appearance to compensate for limited supervision.(3) Our model can be learned with a mixture of weakly labelled and unlabelled data, allowing the large volume of unlabelled images on the Internet to be exploited for learning. Extensive experiments on the challenging VOC dataset demonstrate that our approach outperforms the state-of-the-art competitors.

Efficient Image and Video Co-localization with Frank-Wolfe Algorithm

“…We also combine the two and present results for joint image-video co-localization. Following previous works in weakly supervised localization (WSL) [13,17,32,[43][44][45][46] and co-localization [47], we use the CorLoc evaluation metric, defined as the percentage of images correctly localized according to the PASCAL-criterion: area(Bp∩Bgt) area(Bp∪Bgt) > 0.5, where B p is the predicted box and B gt is the ground-truth box. All CorLoc results are given in percentages.…”

“…We also show our method compared to previous WSL methods in Table 3 for the PASCAL07 dataset [13]. In the WSL scenario, previous methods are also given negative data [13,17,32,[43][44][45][46], whereas our method is not. Here, we can see that although our method is able to work Table 4.…”

“…Our method (Co-localization) [44] (WSL) [46] (WSL) [45] (WSL) [43] (WSL) [17] Table 3. CorLoc results on PASCAL07 compared to previous methods for weakly supervised localization (WSL).…”

“…The co-localization problem is similar to co-segmentation [21-24, 38, 39, 50] and weakly supervised localization (WSL) [13,17,31,32,[43][44][45][46]49]. In contrast to co-segmentation, we seek to localize objects with bounding boxes rather than segmentations, which allows us to greatly decrease the number of variables in our problem.…”

“…In addition to pixels, these images and videos are often tagged with the visual concepts they contain, leading to a natural source of weakly labeled data. Recent research has studied ways of leveraging this data, such as weakly supervised localization [13,17,32,[43][44][45][46], co-segmentation [21,23,38], and co-localization [37,47].…”