Sensing through structure

Abstract

We present an approach to designing input devices that focuses on the structure of materials. We explore and visualize how a material reacts under manipulation, and harness the material's properties to design new movement sensors. Two benefits spring out of this approach. One, simpler sensing emerges from making use of existing structure in the material. Two, by working with the natural structure of the material, we create input devices with readily recognizable affordances. We present six projects using this approach. We use the natural structure (coordination) of the human body to enable a mapping from five clothing-mounted accelerometers to high-quality motion capture data, creating a low-cost performance animation system. We design silicone input devices with embedded texture allowing single-camera tracking. We study squishable, conformable materials such as foam and silicone, and create a vocabulary of unit structures (shaped cuts in the material) for harnessing patterns of compression/tension to capture particular manipulations. We use this vocabulary to build soft sensing skeletons for stuffed animals, making foam cores with e-textile versions of our unit structures. We also use this vocabulary to design a tongue input device for a collaboration with Disney Imagineering. Finally, we rethink this vocabulary and apply it to capturing, using air pressure sensors, manipulations of hollow 3D-printed rubber shapes, and 3D-print several interactive robots incorporating the new vocabulary.

Technofuturism in Play

“…Not only does Disney shy away from promoting many of the technologies employed to achieve objectives associated with their tracking agenda, but they frame many of these systems using somewhat defensive language within their attempts to convince visitors of the advantages and fend off criticism from media sources. For example, Disney has never explicitly blogged about step trackingwhich is done through MagicBands 13or embedded sensors within floors or walkways, yet various sources, including publications by Disney "imagineers," assert they engage in these practices and have tested a number of different approaches (e.g., Andrews et al 2016;Slyper, Poupyrev, and Hodgins 2010). Instead, Disney glosses over them, while introducing the features or attractions that these forms of tracking support, mentioning the underlying technologies only in the vaguest terms, yet with highly positive language and framing.…”

MetaSense: Integrating Sensing Capabilities into Mechanical Metamaterial

“…More recently, conductive flament has been used to detect object deformation. While prior research created deformation sensors by casting silicone [8,35] or attaching copper tape to shear structures (Foldio [21]), researchers showed that deformation sensing can also be achieved by printing parts of the object with conductive flament (Flexibles [30]).…”

Flexibles

“…Deformation sensing can be achieved by embedding sensors into objects [27,47,53,49,30,48] or using optical sensing [12,9,46,45,33,54]. Other approaches employ resistive [44,10,1], capacitive [28], or piezoelectric [35,36] sensing.…”

“…Probably most closely related to our approach is work by Slyper et al [44] and more recently Bächer et al [1]. Slyper et al embed wires inside manually fabricated soft silicone objects of varied geometry to resistively or magnetically sense versatile interactions, including bending, twisting, pressing, and stretching.…”

“…Prior approaches capture deformation input using camera-based touch tracking [12,9], but require a stationary tracking solution in contrast to the now more commonplace and mobile capacitive touchscreens. Recent approaches utilize resistive [44,49,36,1], capacitive [28], or pneumatic [50] sensing, but require an active object with embedded or tethered electronics and a power supply. This makes them less attractive for use as tangible objects on today's capacitive touchscreens.…”