Figure 1. A range of textile-embedded 3D printed objects fabricated using our techniques-a box with a rolling lid containing a mesh of polyester and strings for actuation (a); a functional watchband printed on a polyester mesh (b); a figure with a pressure-sensitive head that controls an embedded displacement sensor containing a mesh of nylon and spandex fibers (c); a 22 inch (56 cm) crown printed on a single piece of felt larger than the print bed (d).

Blind people frequently encounter inaccessible dynamic touchscreens in their everyday lives that are difficult, frustrating, and often impossible to use independently. Touchscreens are often the only way to control everything from coffee machines and payment terminals, to subway ticket machines and in-flight entertainment systems. Interacting with dynamic touchscreens is difficult non-visually because the visual user interfaces change, interactions often occur over multiple different screens, and it is easy to accidentally trigger interface actions while exploring the screen. To solve these problems, we introduce StateLens -a three-part reverse engineering solution that makes existing dynamic touchscreens accessible. First, StateLens reverse engineers the underlying state diagrams of existing interfaces using point-of-view videos found online or taken by users using a hybrid crowd-computer vision pipeline. Second, using the state diagrams, StateLens automatically generates conversational agents to guide blind users through specifying the tasks that the interface can perform, allowing the StateLens iOS application to provide interactive guidance and feedback so that blind users can access the interface. Finally, a set of 3Dprinted accessories enable blind people to explore capacitive touchscreens without the risk of triggering accidental touches on the interface. Our technical evaluation shows that State-Lens can accurately reconstruct interfaces from stationary, hand-held, and web videos; and, a user study of the complete system demonstrates that StateLens successfully enables blind users to access otherwise inaccessible dynamic touchscreens.

In this paper, we examine the future of designing room-scale deployable pneumatic structures that can be fabricated with interactive capabilities and thus be responsive to human input and environments. While there have been recent advances in fabrication methods for creating large-scale structures, they have mainly focused around creating passive structures. Hence in this work, we collectively tackle three main challenges that need to be solved for designing room-scale interactive deployable structures namely -- the input, output (actuation) and construction methods. First, we explore three types of sensing methods --- acoustic, capacitive and pressure --- in order to embed input into these structures. These sensing methods enable users to perform gestures such as knock, squeeze and swipe with specific parts of our fabricated structure such as doors, windows, etc. and make them interactive. Second, we explore three types of actuation mechanisms -- inflatable tendon drive, twisted tendon drive and roll bending actuator -- that are implemented at structural scale and can be embedded into our structures to enable a variety of responsive actuation. Finally, we provide a construction method to custom fabricate and assemble inter-connected pneumatic trusses with embedded sensing and actuation capability to prototype interactions with room-scale deployable structures. To further illustrate the collective (input, output and construction) usage of the system, we fabricated three exemplar interactive deployable structures -- a responsive canopy, an interactive geodesic dome and a portable table (Figures 1 and 2). These can be deployed from a compact deflated state to a much larger inflated state which takes on a desired form while offering interactivity.

No abstract

Figure 1: A range of objects fabricated on our novel 3D printer using rigid plastic and electrospun textiles: (A) a close-up of our printer electrospinning; (B) an origami-style folding lamp with piezoresistve brightness control and a soft custom-shaped capacitive toggle switch that use electrospun textiles for sensing and tactile experiences; (C) an actuated electrospun and rigid plastic fower that opens when an electrospun textile liquid sensor detects sufcient water in the soil; (D) a sheep comic that uses electrospun textile with capacitive sensing to create an interactive tactile experience.