Characterisation and adaptive learning in interactive video retrieval

Abstract

Retrieving videos by content is a very challenging task because it involves a wide variety of fields. From low-level video descriptors to high-level visual understanding, Content-based Video Retrieval (CBVR) systems have to fill a huge semantic gap to provide users with those videos which satisfy their queries. Even though some of the state-of-the-art approaches have shown to be successful on reduced databases, the ongoing expansion of video collections demands new capabilities in CBVR. Retrieval systems are required to be more efficient to deal with this increasing amount of samples and more effective to cope with more complex query concepts. In this thesis, we explore how difficult this task is and how our contributions try to improve the current state-of-the-art.In this work, we are interested in the use of latent topics to overcome the current limitations in CBVR. Despite the potential of topic models to uncover the hidden structure of a collection, they have traditionally been unable to provide a competitive advantage in CBVR because of the high computational cost of their algorithms and the complexity of the latent space in the visual domain. Throughout this thesis we focus on designing new models and tools based on topic models to take advantage of the latent space in CBVR. Specifically, we have worked in four different areas within the retrieval process: vocabulary reduction, encoding, modelling and ranking, being our most important contributions related to both modelling and ranking.Initially, we present a novel approach to vocabulary reduction based on latent topics in order show how topic models are able to capture the more relevant words of a collection. Subsequently, a new encoding approach specially designed to Content-Based Retrieval tasks is proposed. In the modelling stage, we study how the use of different topic models affects video retrieval performance and present an incremental topic model to cope with incremental scenarios in an effective and efficient way. Regarding the ranking stage, we propose a new proba-3 4 Abstract bilistic ranking function which is deduced from a supervised topic model to tackle the semantic gap between low-level features and high-level concepts through the patterns defined by topics. Finally, we conclude the work with observations on how this investigation has impacted the use of topic models in CBVR. AcknowledgmentsI would like to start by thanking my thesis supervisor Prof. Filiberto Pla for having given me the opportunity to reach this goal and, of course, for having shared with me his brilliant knowledge and ideas. Although obvious, I find it essential to remark that this thesis would have not been possible without him.I would also like to thank all the people who have helped me to shape this work during these years. First, thanks to my colleagues at University Jaume I for your useful comments and suggestions. Thanks as well to my colleagues at Bristol University where I had a short but fruitful research stay. Last but not least, I would like to thank my...