Improving Query and Assessment Quality in Text-Based Interactive Video Retrieval Evaluation

Bailer, Werner; Arnold, Rahel; Benz, Vera; Coccomini, Davide; Gkagkas, Anastasios; Guðmundsson, Gylfi Þór; Heller, Silvan; Jónsson, Björn Þór; Lokoc, Jakub; Messina, Nicola; Pantelidis, Nick; Wu, Jiaxin

doi:10.1145/3591106.3592281

Cited by 2 publications

(1 citation statement)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Six judges participated in the sessions. More details and the evolution of this process can be found in [9].…”

Section: Lessons From Text Query Definitionmentioning

confidence: 99%

Interactive video retrieval in the age of effective joint embedding deep models: lessons from the 11th VBS

Lokoč,

Andreadis,

Bailer

et al. 2023

Multimedia Systems

Self Cite

View full text Add to dashboard Cite

This paper presents the findings of the eleventh Video Browser Showdown competition, where sixteen teams competed in known-item and ad-hoc search tasks. Many of the teams utilized state-of-the-art video retrieval approaches that demonstrated high effectiveness in challenging search scenarios. In the paper, a broad survey of all utilized approaches is presented in connection with an analysis of the performance of participating teams. Specifically, both high-level performance indicators are presented with overall statistics as well as an in-depth analysis of the performance of selected tools implementing result set logging. The analysis reveals evidence that the CLIP model represents a versatile tool for cross-modal video retrieval when combined with interactive search capabilities. Furthermore, the analysis investigates the effect of different users and text query properties on the performance in search tasks. Last but not least, lessons learned from search task preparation are presented, and a new direction for adhoc search based tasks at Video Browser Showdown is introduced.

show abstract

“…Six judges participated in the sessions. More details and the evolution of this process can be found in [9].…”

Section: Lessons From Text Query Definitionmentioning

confidence: 99%

Interactive video retrieval in the age of effective joint embedding deep models: lessons from the 11th VBS

Lokoč,

Andreadis,

Bailer

et al. 2023

Multimedia Systems

Self Cite

View full text Add to dashboard Cite

show abstract

Evaluating Performance and Trends in Interactive Video Retrieval: Insights From the 12th VBS Competition

Vadicamo,

Arnold,

Bailer

et al. 2024

IEEE Access

View full text Add to dashboard Cite

This paper conducts a thorough examination of the 12th Video Browser Showdown (VBS) competition, which is a well-established international benchmarking campaign for interactive video search systems. The annual VBS competition has witnessed a steep rise in the popularity of multimodal embedding-based approaches in interactive video retrieval. The majority of the thirteen systems participating in VBS 2023 utilized a CLIP-based cross-modal search model, allowing the specification of free-form text queries to search visual content. This shared emphasis on joint embedding models contributed to balanced performance across various teams. However, the distinguishing factors of the top-performing teams included the adept combination of multiple models and search modes, along with the capabilities of interactive interfaces to facilitate and refine the search process. Our work provides an overview of the state-of-the-art approaches employed by the participating systems and conducts a thorough analysis of their search logs, which record user interactions and results of their queries for each task. Our comprehensive examination of the VBS competition offers assessments of the effectiveness of the retrieval models employed, the browsing efficiency, and user query patterns. Additionally, it provides valuable insights into the evolving landscape of interactive video retrieval and its future challenges.

show abstract

Improving Query and Assessment Quality in Text-Based Interactive Video Retrieval Evaluation

Cited by 2 publications

References 9 publications

Interactive video retrieval in the age of effective joint embedding deep models: lessons from the 11th VBS

Interactive video retrieval in the age of effective joint embedding deep models: lessons from the 11th VBS

Evaluating Performance and Trends in Interactive Video Retrieval: Insights From the 12th VBS Competition

Contact Info

Product

Resources

About