Scene Discovery by Matrix Factorization

Loeff, Nicolas; Farhadi, Ali

doi:10.1007/978-3-540-88693-8_33

Cited by 30 publications

(58 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast, our key insight is that rich contextual information is available in "how" human-provided tags or descriptions are given, specifically in their order, rank, and proximity. Thus, whereas prior work learning representations with accompanying text associates image content with an appropriate word or distribution of words (Qi et al 2009;Makadia et al 2008;Quattoni et al 2007;Loeff and Farhadi 2008;Bekkerman and Jeon 2007;Hardoon and ShaweTaylor 2003;Yakhnenko and Honavar 2009), the semantic space we discover also preserves the relative significance of the objects present. We expect this difference to be most useful in retrieval applications where one wants to access scenes that are perceptually similar though not visually identical, or in auto-tagging applications where very compact focused descriptions are required.…”

Section: Related Workmentioning

confidence: 95%

“…A number of learning strategies have been explored in the literature along these lines, including variants of metric learning (Qi et al 2009;Makadia et al 2008), transfer learning (Quattoni et al 2007), matrix factorization (Loeff and Farhadi 2008), and random field models (Bekkerman and Jeon 2007). Most relevant to our method, some previous work has specifically considered KCCA for this purpose (Hardoon and Shawe-Taylor 2003;Yakhnenko and Honavar 2009;Blaschko and Lampert 2008).…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Learning the Relative Importance of Objects from Tagged Images for Retrieval and Cross-Modal Search

2011

View full text Add to dashboard Cite

We introduce an approach to image retrieval and auto-tagging that leverages the implicit information about object importance conveyed by the list of keyword tags a person supplies for an image. We propose an unsupervised learning procedure based on Kernel Canonical Correlation Analysis that discovers the relationship between how humans tag images (e.g., the order in which words are mentioned) and the relative importance of objects and their layout in the scene. Using this discovered connection, we show how to boost accuracy for novel queries, such that the search results better preserve the aspects a human may find most worth mentioning. We evaluate our approach on three datasets using either keyword tags or natural language descriptions, and quantify results with both ground truth parameters as well as direct tests with human subjects. Our results show clear improvements over approaches that either rely on image features alone, or that use words and image features but ignore the implied importance cues. Overall, our work provides a novel way to incorporate high-level human perception of scenes into visual representations for enhanced image search.

show abstract

Section: Related Workmentioning

confidence: 95%

Section: Related Workmentioning

confidence: 99%

Learning the Relative Importance of Objects from Tagged Images for Retrieval and Cross-Modal Search

2011

View full text Add to dashboard Cite

show abstract

“…Our approach is inspired by the hypothesis speculated in [14] that there exists a latent low-dimensional feature space that are shared by classifiers for different tags. In [14], a low-rank matrix factorization approach is used to exploit this low-dimensional space. In our work, we directly use the latent topics as the low-dimensional space shared by tag classifiers.…”

Section: Mmlda Amentioning

confidence: 99%

Max-margin Latent Dirichlet Allocation for Image Classification and Annotation

Yang

Mori

2011

Procedings of the British Machine Vision Conference 2011

View full text Add to dashboard Cite

We present the max-margin latent Dirichlet allocation, a max-margin variant of supervised topic models, for image classification and annotation. Our model for image classification (called MMLDA c ) integrates discriminative classification with generative topic models. Our model for image annotation (called MMLDA a ) extends MMLDA c to the case of multi-label problems, where each image can be associated with more than one annotation terms. We derive efficient learning algorithms for both models and demonstrate experimentally the advantages of our proposed models over other baseline methods.

show abstract

“…This is because MTL would be able to estimate a latent feature representation shared across all views. While MTL is well known to vision [20,13,16,31], it has never been used for view-invariant action recognition.…”

Section: Introductionmentioning

confidence: 99%

Latent Multitask Learning for View-Invariant Action Recognition

Mahasseni

Todorovic

2013

2013 IEEE International Conference on Computer Vision

View full text Add to dashboard Cite

This paper presents an approach to view-invariant action recognition, where human poses and motions exhibit large variations across different camera viewpoints. When each viewpoint of a given set of action classes is specified as a learning task then multitask learning appears suitable for achieving view invariance in recognition. We extend the standard multitask learning to allow identifying: (1) latent groupings of action views (i.e., tasks), and (2) discriminative action parts, along with joint learning of all tasks. This is because it seems reasonable to expect that certain distinct views are more correlated than some others, and thus identifying correlated views could improve recognition. Also, part-based modeling is expected to improve robustness against self-occlusion when actors are imaged from different views. Results on the benchmark datasets show that we outperform standard multitask learning by 21.9%, and the state-of-the-art alternatives by 4.5-6%.

show abstract

Scene Discovery by Matrix Factorization

Cited by 30 publications

References 20 publications

Learning the Relative Importance of Objects from Tagged Images for Retrieval and Cross-Modal Search

Learning the Relative Importance of Objects from Tagged Images for Retrieval and Cross-Modal Search

Max-margin Latent Dirichlet Allocation for Image Classification and Annotation

Latent Multitask Learning for View-Invariant Action Recognition

Contact Info

Product

Resources

About