In-cAR Gaming: Exploring the use of AR headsets to Leverage Passenger Travel Environments for Mixed Reality Gameplay

Abstract

Autonomous cars offer passengers a rich platform for Augmented Reality entertainment, with complex sensing that can drive passenger experiences by tracking, appropriating and altering elements of reality. This paper forms an early exploration of in-car AR games, starting with how existing game genres might work within an AR vehicle context, appropriating elements of reality (e.g. other cars) into gameplay, and altering the appearance of reality in relation to game events (e.g. augmented cracks in car windows). We discuss results from focus groups exploring an initial AR game prototype, and an informal in-car evaluation of a follow-up prototype inspired by the focus groups. Broadly, we found that participants enjoyed using AR gaming in-car, noted the immersive impact of appropriating real-world elements into gameplay, and felt that it improved their experience of the journey. We reflect on the ways in which future AR passenger experiences might take advantage of the available sensing and environment to create engaging reality-based gameplay. CCS CONCEPTS• Human-centered computing → Mixed / augmented reality.

An investigation into generation Z’s mindsets of entertainment in an autonomous vehicle

“…Integrating brought-in devices within automobiles, and connecting and syncing information, has upgraded the in-car entertainment experience [37,38]. Today, the paradigm shift through automation in car systems enables non-driving related tasks with relevant entertainment systems such as immersive media, games, comprehensive video streaming platforms, social networking and AR/VR or mixed reality interactions [39,40,41,42,43].…”

PassengXR: A Low Cost Platform for Any-Car, Multi-User, Motion-Based Passenger XR Experiences

“…As well as providing more opportunities for entertainment [17,18,48], XR also offers passengers significant benefits for productivity, as workspaces are not restricted to the physical dimensions of desks or monitors. Research has begun investigating the effective, comfortable and socially acceptable design of virtual workspaces in XR [32] including in cars [28] and planes [37], letting passengers interact with multiple windows from a seated position while limiting head movements.…”

“…Location-based experiences (LBEs) are a key component of many envisaged passenger XR scenarios, for example augmented tourism [3], location-based gaming [48] etc. To enact an LBE, we need to know where in the world the vehicle is over time, and in Passen-gXR this is established using GNSS positioning.…”

“…Supported XR Headsets We have so far focused on support for -and testing of -VR headsets, as they have wider uptake in commercial and research passenger XR endeavours. However, AR has a long history in supporting drivers and recent research has suggested that the technology is also well-suited to passenger experiences [48,50]. Unity can build applications for Windows Mixed Reality (WMR) platforms, and we are currently in the process of testing PassengXR with AR headsets such as Hololens 18 and NReal 19 , and we anticipate that our proposed solutions will in time work on these platforms.…”

“…LiDAR sensing). However, potential users of our platform can utilize any Unity-compatible sensing to e.g., provide support for detecting and appropriating real objects around the car for engaging XR experiences, such as demonstrated by Togwell et al [48] and their use of the ZED2 camera 20 to track other vehicles.…”