Figure 1: inFORM enables new interaction techniques for shape-changing UIs. Left to right: On-demand UI elements through Dynamic Affordances; Guiding interaction with Dynamic Constraints; Object actuation; Physical rendering of content and UI. ABSTRACTPast research on shape displays has primarily focused on rendering content and user interface elements through shape output, with less emphasis on dynamically changing UIs. We propose utilizing shape displays in three different ways to mediate interaction: to facilitate by providing dynamic physical affordances through shape change, to restrict by guiding users with dynamic physical constraints, and to manipulate by actuating physical objects. We outline potential interaction techniques and introduce Dynamic Physical Affordances and Constraints with our inFORM system, built on top of a stateof-the-art shape display, which provides for variable stiffness rendering and real-time user input through direct touch and tangible interaction. A set of motivating examples demonstrates how dynamic affordances, constraints and object actuation can create novel interaction possibilities.

Actuated shape output provides novel opportunities for experiencing, creating and manipulating 3D content in the physical world. While various shape displays have been proposed, a common approach utilizes an array of linear actuators to form 2.5D surfaces. Through identifying a set of common interactions for viewing and manipulating content on shape displays, we argue why input modalities beyond direct touch are required. The combination of freehand gestures and direct touch provides additional degrees of freedom and resolves input ambiguities, while keeping the locus of interaction on the shape output. To demonstrate the proposed combination of input modalities and explore applications for 2.5D shape displays, two example scenarios are implemented on a prototype system.

Figure 1. ShapeBots exemplifies a new type of shape-changing interface that consists of a swarm of self-transformable robots. A) Two ShapeBot elements. B) A miniature reel-based linear actuator for self-transformation. By leveraging individual and collective transformation, ShapeBots can provide C) interactive physical display (e.g., rendering a rectangle), D) object actuation (e.g., cleaning up a desk), E) distributed shape display (e.g., rendering a dynamic surface), and F) embedded data physicalization (e.g., showing populations of states on a US map). ABSTRACTWe introduce shape-changing swarm robots. A swarm of selftransformable robots can both individually and collectively change their configuration to display information, actuate objects, act as tangible controllers, visualize data, and provide physical affordances. ShapeBots is a concept prototype of shape-changing swarm robots. Each robot can change its shape by leveraging small linear actuators that are thin (2.5 cm) and highly extendable (up to 20cm) in both horizontal and vertical directions. The modular design of each actuator enables various shapes and geometries of self-transformation. We illustrate potential application scenarios and discuss how this type of interface opens up possibilities for the future of ubiquitous and distributed shape-changing interfaces.interfaces [5,38]-will follow the same path as technology advances. Although current interfaces are often large, heavy, and immobile, these interfaces will surely be replaced with hundreds of distributed interfaces, in the same way that desktop computers were replaced by hundreds of distributed mobile computers. If shape-changing interfaces will become truly ubiquitous, how can these interfaces be distributed and embedded into our everyday environment? This paper introduces shape-changing swarm robots for distributed shape-changing interfaces. Shape-changing swarm robots can both collectively and individually change their shape, so that they can collectively present information, act as controllers, actuate objects, represent data, and provide dynamic physical affordances.

Figure 1. RoomShift is composed of a swarm of shape-changing robots for haptic feedback in VR. RoomShift robots move beneath a piece of furniture to lift, move and place it. Multiple robots move furniture to construct a physical haptic environment collectively. The corresponding virtual scene is shown, with a human silhouette added for a reference.

Malleable and organic user interfaces have the potential to enable radically new forms of interactions and expressiveness through flexible, free-form and computationally controlled shapes and displays. This work, specifically focuses on particle jamming as a simple, effective method for flexible, shape-changing user interfaces where programmatic control of material stiffness enables haptic feedback, deformation, tunable affordances and control gain. We introduce a compact, low-power pneumatic jamming system suitable for mobile devices, and a new hydraulic-based technique with fast, silent actuation and optical shape sensing. We enable jamming structures to sense input and function as interaction devices through two contributed methods for high-resolution shape sensing using: 1) index-matched particles and fluids, and 2) capacitive and electric field sensing. We explore the design space of malleable and organic user interfaces enabled by jamming through four motivational prototypes that highlight jamming's potential in HCI, including applications for tabletops, tablets and for portable shape-changing mobile devices.