The future of ‘video’ in video-based qualitative research is not ‘dumb’ flat pixels! Exploring volumetric performance capture and immersive performative replay

Abstract

Qualitative research that focuses on social interaction and talk has been increasingly based, for good reason, on collections of audiovisual recordings in which 2D flat-screen video and mono/stereo audio are the dominant recording media. This article argues that the future of ‘video’ in video-based qualitative studies will move away from ‘dumb’ flat pixels in a 2D screen. Instead, volumetric performance capture and immersive performative replay rely on a procedural camera/spectator-independent representation of a dynamic real or virtual volumetric space over time. It affords analytical practices of re-enactment – shadowing or redoing modes of seeing/listening as an active spectation for ‘another next first time’ – which play on the tense relationships between live performance, observability, spectatorship and documentation. Three examples illustrate how naturally occurring social interaction and settings can be captured volumetrically and re-enacted immersively in virtual reality (VR) and what this means for data integrity, evidential adequacy and qualitative analysis.

LightNet: a networking library to perform multi-participant experiments in Virtual Reality

“…More specifically, participants are able to immerse into the virtual environment, that is, they "become active participants [...] seeing, hearing, feeling, and actuating as if they were in the real world" (11), with the feeling of interacting with others during a virtual social encounter being comparable to everyday encounters with participants able to access their affective and cognitive states (14)(15)(16)(17). As a result, when investigating social interaction in VR, it becomes possible to combine the advantages of VR in regards to ecological validity also in respect to the social domain, with the high controllability of virtual experimental environments (18). Moreover, measuring two people in VR could account for the multi-modality of joint action.…”

“…This is particularly interesting for experiments where participants should freely examine a virtual environment while tracking and assessing their gaze direction as a function of time. Moreover, while traditional experiments have to re-play the same social situation with other actors and environments, VR scenarios can be exactly replayed within and across experiments, significantly reducing their variability and minimizing distracting influences like experimenter bias (18,36). Another important factor in social interaction research is the amount of real-time interaction included in experimental designs.…”

“…Hence, during a virtual social encounter, one feels like interacting with others just as in every-day encounters being able to access their affective and cognitive states (14)(15)(16)(17). Simultaneously, experimental conditions are less variable due to the high degree of control offered by experimental environments that have been created virtually (18). Even non-verbal cues such as interpersonal distance, body posture, and facial expression can be captured precisely (19)(20)(21)(22).…”

Kollaborativ læring i social 360VR

“…Med 360° video er undervisere og studerende ikke laengere tvunget til at se og interagere med video på en flad skaerm, i stedet kan 360° video overføres til og afspilles i et Head Mounted Display/VR-briller (HMD). Deltagere er dermed ikke begraenset til et udsnit af virkeligheden, men kan opleve en mere rummelig gengivelse af en optaget situation (McIlvenny, 2020). Dette skaber nye muligheder for situerede laeringsforståelser, da en laeringssituation vil kunne fastholdes i sin rumlige helhed og skabe bedre deltagelsesmuligheder for de laerende (Lave & Wenger, 2003).…”

Entire issue

“…In this paper, we explore how 360 video can be used in clinical exercises in Higher Education (HE) medical education as an extension of case-PBL (Stentoft, 2019). With social 360VR teachers and students are no longer confined to viewing a video on a flat screen, instead the 360 video is projected in a Head Mounted Display (HMD) (McIlvenny, 2020a) providing a more immersive social experience. With social 360°VR new opportunities for bringing authentic cases into PBL arise and new formats for collaboration is emerging .…”