User-Driven Constraints for Layout Optimisation in Augmented Reality

Abstract

Automatic layout optimisation allows users to arrange augmented reality content in the real-world environment without the need for tedious manual interactions. This optimisation is often based on modelling the intended content placement as constraints, defined as cost functions. Then, applying a cost minimization algorithm leads to a desirable placement. However, such an approach is limited by the lack of user control over the optimisation results. In this paper we explore the concept of user-driven constraints for augmented reality layout optimisation. With our approach users can define and set up their own constraints directly within the real-world environment. We first present a design space composed of three dimensions: the constraints, the regions of interest and the constraint parameters. Then we explore which input gestures can be employed to define the user-driven constraints of our design space through a user elicitation study. Using the results of the study, we propose a holistic system design and implementation demonstrating our user-driven constraints, which we evaluate in a final user study where participants had to create several constraints at the same time to arrange a set of virtual contents. CCS CONCEPTS• Human-centered computing → Interactive systems and tools.

Towards Flexible and Robust User Interface Adaptations With Multiple Objectives

“…Instead, it can be applied to a wide range of UI optimization problems where multiple, potentially conflicting objectives need to be considered. In the context of 3D UI adaptation, other objectives related to, for example, visibility [3,13,31,36], reachability [3,13,15], semantics [13,36], spatio-temporal consistency [3], user-specified preference regions [36], or cognitive load [31] may be explored. Similarly, alternative design variables, for example related to a UI's level of detail [31], may be considered.…”

“…Starting with the ergonomic objectives discussed in Section 3, we add two further objectives that leverage the relation between the adapted UI and the virtual and physical objects in the user's environment. These two objectives are inspired by the semantic objectives in [13] and [36] and constitute both a Semantic "Pull", minimizing the distance between the UI and the most strongly positively associated anchor object in the user's environment, and a Semantic "Push", maximizing the distance between the UI and the most strongly negatively associated anchor object in the user's environment. 2 For this example, we manually define positive and negative association weights for the objects in the example environment.…”

“…These utility functions reflect qualities of real end-user preferences and are used to quantify an adaptation's fit to a simulated user's preferences. They are inspired by adaptation criteria from prior literature and include both workplace ergonomic metrics [3,15,35] (i.e., a neck, a shoulder, and a reachability objective) and semantic relatedness measures [13,36] which are used to create a set of approximately naturalistic satisfaction criteria (see Appendix A.1). We parametrize the utility functions with a uniform probability distribution over their weight parameters to represent the variety and individuality of preferences that may be present in real end-user populations (cf.…”