Automatic Generation of User Interface Layouts for Alternative Screen Orientations

Abstract

Abstract. Creating multiple layout alternatives for graphical user interfaces to accommodate different screen orientations for mobile devices is labor intensive. Here, we investigate how such layout alternatives can be generated automatically from an initial layout. Providing good layout alternatives can inspire developers in their design work and support them to create adaptive layouts. We performed an analysis of layout alternatives in existing apps and identified common realworld layout transformation patterns. Based on these patterns we developed a prototype that generates landscape and portrait layout alternatives for an initial layout. In general, there is a very large number of possibilities of how widgets can be rearranged. For this reason we developed a classification method to identify and evaluate "good" layout alternatives automatically. From this set of "good" layout alternatives, designers can choose suitable layouts for their applications. In a questionnaire study we verified that our method generates layout alternatives that appear well structured and are easy to use.

Responsive and Personalized Web Layouts with Integer Programming

“…Integer programming (IP) is a mathematical optimization method that has been widely adopted for computational generation and adaptation of graphical user interfaces (GUI) [28]. It has been successfully applied for generating GUIs automatically for multiple devices [14] and screen orientations [45], learning cost functions for user interaction -based GUI optimization [15], designing GUIs via high-level constraints [19], optimizing GUIs for aesthetics [44], and generating alternative layouts for design exploration [11]. In IP, a problem is defined as a set of equality or inequality constraints on decision variables, such as elements' coordinates in a layout.…”

Combinatorial Optimization of Graphical User Interface Designs

“…3) Adaptation and Personalization: Combinatorial optimization can adapt designs when its objective function is able to represent individual-specific or moment-bymoment requirements. For example, a design can be reoptimized when the device orientation changes [85]. Todi et al [79] adapted web layouts in line with the visual history of a user [80], and Lindlbauer et al [86] proposed an algorithm to automatically adapt mixed reality interfaces in line with the current user context, leveraging the environment and the cognitive load imposed by the user's current task.…”

“…Quality ratings involve questionnaire-based studies that can be run for subjectively assessing the quality of the designs produced. In a rating-based study with software developers (N = 20), portrait/landscape transformations of GUIs [85] were performed by an optimizer using design heuristics for objectives. No statistical testing was utilized.…”

Robust relational layout synthesis from examples for Android

“…Given the complexity and time requirements of this task, a common method is to outsource the task to a company that manually creates layouts from images for a high fee [apptype 2018; psd2android 2018; psd2mobi 2018; replia 2018]. Another approach is to try and automate part of the process using better tool support for layout design [Zeidler et al 2013a,b], generating user interface sketches from real images [Swearngin et al 2018] or hand drawings [Corrado et al 2018], sketch based code search for similar layouts [Reiss 2014] or generation of layouts for different screen orientations [Zeidler et al 2017].…”