User Interaction for Visual Lifelog Retrieval in a Virtual Environment

Abstract

Efficient retrieval of lifelog information is an ongoing area of research due to the multifaceted nature, and ever increasing size of lifelog datasets. Previous studies have examined lifelog exploration on conventional hardware platforms, but in this paper we describe a novel approach to lifelog retrieval using virtual reality. The focus of this research is to identify what aspects of lifelog retrieval can be effectively translated from a conventional to a virtual environment and if it provides any benefit to the user. The most widely available lifelog datasets for research are primarily image-based and focus on continuous capture from a first-person perspective. These large image corpora are often enhanced by image processing techniques and various other metadata. Despite the rapidly maturing nature of virtual reality as a platform, there has been very little investigation into user interaction within the context of lifelogging. The experiment outlined in this work seeks to evaluate four different virtual reality user interaction approaches to lifelog retrieval. The prototype system used in this experiment also competed at the Lifelog Search Challenge at ACM ICMR 2018 where it ranked first place.

Exploring Intuitive Lifelog Retrieval and Interaction Modes in Virtual Reality with vitrivr-VR

“…In fact, the very first winner of the LSC [4] -the Virtual Reality Lifelog Explorer (VRLE) -featured a VR UI and has been a participant again in 2020 [5]. While one of the first and most effective VR lifelog exploration systems, VRLE is a rather conservative VR adaptation of a conventional 2D user interface and even though different modes of user interaction were explored in [2], the query menu and horizontally scrolling result display are very similar to conventional UIs and do not take full advantage of the virtual space.…”

Reflecting on Emotions within VR Mood Worlds

“…Although it is mostly known for anxiety or exposure therapy [25], it has also been used in a variety of other mental health related areas, e.g. mindfulness [27], stress reduction [40,45], visual lifelogging [12], or to recreate memories [39,41]. Additionally, it is also becoming more prominent for ER interventions [26].…”

PMData: A sports logging dataset

“…Subjective assessments of each participant's wellness, training load, and injuries have been logged using the PM Reporter Pro smartphone application 4 where Figure 2 shows an example of a reporting sequence. PM Reporter is part of the PMSys online sports logging system that enables athletes to monitor individual training load, daily subjective wellness parameters, and injuries [20].…”

“…Some work has been done with similar datasets, for example, retrieving moment of interest [5,15]. However, a key challenge in lifelogging research is the poor availability of test collections [4]. Hence, there is a need for more available lifelog datasets.…”