Multimodal ranking for image search on community databases

Richter, Fabian; Romberg, Stefan; Hörster, Eva; Lienhart, Rainer

doi:10.1145/1743384.1743402

Cited by 22 publications

(11 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Image retrieval is then formulated as a random walk process. For example, visual ranking algorithms are proposed in [22], [23] to apply PageRank [24] as a solution to large scale image search. By introducing multiple types of edges, graph-based image retrieval can be improved by fusing multiple visual feature modalities and exploiting their mutual reinforcement [25].…”

Section: Related Workmentioning

confidence: 99%

Graph-Based Topic-Focused Retrieval in Distributed Camera Network

Jagadeesh

et al. 2013

IEEE Trans. Multimedia

View full text Add to dashboard Cite

Abstract-Wide-area wireless camera networks are being increasingly deployed in many urban scenarios. The large amount of data generated from these cameras pose significant information processing challenges. In this work, we focus on representation, search and retrieval of moving objects in the scene, with emphasis on local camera node video analysis. We develop a graph model that captures the relationships among objects without the need to identify global trajectories. Specifically, two types of edges are defined in the graph: object edges linking the same object across the whole network and context edges linking different objects within a spatial-temporal proximity. We propose a manifold ranking method with a greedy diversification step to order the relevant items based on similarity as well as diversity within the database. Detailed experimental results using video data from a 10-camera network covering bike paths are presented.Index Terms-Distributed camera network, diverse and relevant ranking, graph-based modeling, information search and retrieval.

show abstract

Section: Related Workmentioning

confidence: 99%

Graph-Based Topic-Focused Retrieval in Distributed Camera Network

Jagadeesh

et al. 2013

IEEE Trans. Multimedia

View full text Add to dashboard Cite

show abstract

“…In terms of image ranking, there are many off-the-shelf choices. Previous work by [6,3] have used Pagerank [4] to compute the scores for each image. However, PageRank gives a static scores for each vertex independent of query.…”

Section: Fig 2 Performance Of Hypergraph Ranking and Other Unimodalmentioning

confidence: 99%

“…Besides image visual features, it has been shown in previous work [5] that integrating other modalities can boost retrieval performance. In the work of [6], graphs have been used to convey multimodal information for search and retrieval. However, all of the graph-based approach above use simple graphs, which can only capture pairwise image relations.…”

Section: Introductionmentioning

confidence: 99%

Unified hypergraph for image ranking in a multimodal context

Singh

Guan

et al. 2012

2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Image ranking has long been studied, yet it remains a very challenging problem. Increasingly, online images come with additional metadata such as user annotations and geographic coordinates. They provide rich complementary information. We propose to combine such multimodal information through a unified hypergraph to improve image retrieval performance. Hypergraphs allow for the simultaneously capture of higher order relationships among images using different modalities, e.g. visual content, user tags, and geolocations. Each image is represented as a vertex in the hypergraph. Each hyperedge is formed by a vertex and it's k-nearest neighbors. Three types of hyperedges exist in our unified hypergraph, which are in correspondence to the three different modalities. Image ranking is then formulated as a ranking problem on a unified hypergraph. The proposed method can easily be extended to incorporate additional modalities as long as a similarity function exists to compare the features. Experimental results on large datasets are promising.

show abstract

“…Research works on image and video search [13,9] also make use of multimodal features such as visual features and user tags for object representation. However, in the search application, all the objects in the database are ranked by their distance w.r.t to the querying object, i.e.…”

Section: Related Workmentioning

confidence: 99%

Multimodal ranking for non-compliance detection in retail surveillance

Trinh

Pankanti

Fan

2012

2012 IEEE Workshop on the Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

In retail stores, cashier non-compliance activities at the Point of Sale (POS) are one of the prevalent sources of retail loss. In this paper, we propose a novel approach to reliably rank the list of detected non-compliance activities of a given retail surveillance system, thereby provide a means of significantly reducing the false alarms and improving the precision in non-compliance detection. Our approach represents each detected non-compliance activity using multimodal features coming from video data, transaction logs (TLog) data and intermediate results of the video analytics. We then learn a binary classifier that successfully separate true positives and false positives in a labeled training set. A confidence score for each detection can then be computed using the decision value of the trained classifier, and a ranked list of detections can be formed based on this score. The benefit from having this ranked list is two-fold. First, a large number of false alarms can be avoided by simply keeping the top part of the list and discarding the rest. Second, a trade off between precision and recall can easily be performed by sliding the discarding threshold along this ranked list.Experimental results on a large scale dataset captured from real stores demonstrate that our approach achieves better precision than a state-of-the-art system at the same recall. Our approach can also reach an operating point that exceeds the retailers' expectation in terms of precision, while retaining an acceptable recall of more than 60%.

show abstract

Multimodal ranking for image search on community databases

Cited by 22 publications

References 23 publications

Graph-Based Topic-Focused Retrieval in Distributed Camera Network

Graph-Based Topic-Focused Retrieval in Distributed Camera Network

Unified hypergraph for image ranking in a multimodal context

Multimodal ranking for non-compliance detection in retail surveillance

Contact Info

Product

Resources

About