ViRMA: Virtual Reality Multimedia Analytics at Video Browser Showdown 2022

Abstract

In this paper we describe the first iteration of the ViRMA prototype system, a novel approach to multimedia analysis in virtual reality, that is inspired by the M 3 data model. In this model, media is mapped into a multidimensional space, based on its metadata. ViRMA users can then interact with the media collection by dynamically projecting the metadata space to the 3D virtual space, through a variety of interactions, and exploring the resulting visualisations.

Interactive video retrieval in the age of effective joint embedding deep models: lessons from the 11th VBS

“…Also, Exquisitor and ViRMA both support concept search for 12,988 ImageNet concepts, which were extracted for each keyframe using a pre-trained DCNN ResNet model [59]. To support its browsing and data model in VR, ViRMA further organises these concepts into a hierarchical structure using semantic relationships derived from WordNet [19]. Finally, in addition to the ImageNet concepts, Exquisitor also maintains search for activity concepts of Kinetics-700, extracted from the video shots using a pre-trained 3D-ResNet model [26].…”

“…The ViRMA [19] prototype system employs a novel VR interaction approach by utilizing the M 3 data model [23], which takes the media objects from the VBS dataset and maps them into a multidimensional media space based on their metadata. Users can then visualize the video data by filtering and dynamically projecting the multidimensional media space to the more familiar 3D space and then can explore this visualization using virtual reality [19]. This type of 3D visualization is effective at browsing and summarising a collection, but is less effective at search, which is likely why the ViRMA system did not perform well in VBS 2022.…”