Multimodal Interactive Lifelog Retrieval with vitrivr-VR

Abstract

The multimodal nature of lifelog data poses unique challenges for analysis, indexing and interactive retrieval. To address these challenges, the Lifelog Search Challenge (LSC) is an annual evaluation campaign allowing interactive retrieval systems to explore new ideas and measure their performance against each other.This paper describes the virtual reality (VR) multimedia retrieval system vitrivr-VR, with a focus on aspects relevant to the LSC'22, especially the user interaction in VR, the formulation of typical LSC queries, and different options to explore the retrieval results in VR.

Introduction to the Fifth Annual Lifelog Search Challenge, LSC'22

“…The vitrivr system, a general purpose multimedia data indexing and retrieval framework [8], supported lifelog search using a number of fundamental techniques, such as text embedding, visual similarity search, classical Boolean operators as well as supporting lifelog-specific enhancements such as dynamic result sequencing and faceted filtering. Built on-top of the vitrivr stack was vitrivr-VR [15], which provided a VR-based access mechanism to the lifelog dataset. The vitrivr-VR system for LSC'22 featured a number of enhancements over the LSC'21 system and supported three different interfaces (or views) designed to support lifelog data access.…”