Extensive research has outlined the potential of augmented, mixed, and virtual reality applications. However, little attention has been paid to scalability enhancements fostering practical adoption. In this paper, we introduce the concept of scalable extended reality (XRS), i.e., spaces scaling between different displays and degrees of virtuality that can be entered by multiple, possibly distributed users. The development of such XRS spaces concerns several research fields. To provide bidirectional interaction and maintain consistency with the real environment, virtual reconstructions of physical scenes need to be segmented semantically and adapted dynamically. Moreover, scalable interaction techniques for selection, manipulation, and navigation as well as a world-stabilized rendering of 2D annotations in 3D space are needed to let users intuitively switch between handheld and head-mounted displays. Collaborative settings should further integrate access control and awareness cues indicating the collaborators’ locations and actions. While many of these topics were investigated by previous research, very few have considered their integration to enhance scalability. Addressing this gap, we review related previous research, list current barriers to the development of XRS spaces, and highlight dependencies between them.

Mixed and Virtual Reality technologies have been assigned considerable potential to support training and workflows in various domains. However, available solutions are subject to scalability limitations which evoke temporal and cognitive efforts that outweigh the technology’s intrinsic potential and prevent their application in profit-making, real-world settings. Addressing these issues, we developed a framework for Scalable Extended Reality (XRS) spaces following a human-centered design process. To this end, we derived abstract high-level use cases which exploit key benefits of Mixed and Virtual Reality technologies and can be combined with each other to describe specific low-level use cases in many domains. Based on the defined high-level use cases, i.e., design and development of physical items, training, teleoperation, co-located and distributed collaboration, we specified functional and non-functional requirements and developed a framework design solution that implements multidimensional scalability enhancements: Multiple on-site and off-site users can access the XRS space through customized Mixed or Virtual Reality interfaces and then reference or manipulate real or virtual scene components. Thereby, full scalability regarding options of interaction is provided through the integration of a robotic system that allows off-site users to manipulate real scene components on site. Eventually, the framework’s applicability to different use cases is demonstrated in theoretical walkthroughs.

No abstract