Sensory Augmentation: Toward a Dialogue Between the Arts and Sciences

Abstract

People sense the world by exploiting correlations between their physical actions and the changing sensory input that results from those actions. Interfaces that translate non-human sensor data to signals that are compatible with the human senses can therefore augment our abilities to make sense of the world. This insight has recently sparked an increase in projects that explore sensemaking and the creation of novel human experiences across scientific and artistic disciplines. However, there currently exists no constructive dialogue between artists and scientists that conduct research on this topic. In this position paper, we identify the theory and practice of sensory augmentation as a domain that could benefit from such a dialogue. We argue that artistic and scientific methods can complement each other within research on sensory augmentation and identify six thematic starting points for a dialogue between the arts and sciences. We conducted a case study to explore these conjectures, in which we instigated such a dialogue on a small scale. The case study revealed that the six themes we identified as relevant for a dialogue on sensory augmentation emerge rather spontaneously in such a dialogue and that such an exchange may facilitate progress on questions that are central to the theory and practice of sensory augmentation. Overall, this position paper contributes preliminary evidence for the potential of, and a starting point for, a dialogue between the arts and sciences that advances our understanding of sensory augmentation and the development of applications that involve it.

Bridging a sensory gap between deaf and hearing people–A plea for a situated design approach to sensory augmentation

“…Moreover, existing applications of sensory augmentation technology suffer from technological limitations, such as limitations of the bandwidth of sensory input and of the resolution of actuators, that may not measure up to the sensitivity of the human sense the information is transmitted to (Kristjánsson et al, 2016). Rather than planning for such innovation, however, a situated design approach demands that (de Rooij et al, 2018b) and abstract tasks in lab-situations (Mahatody et al, 2010). The domain of humancentered design offers design methods such as personas and focus groups, but they are specifically used for inquiry about the context of use (Sleeswijk-Visser, 2009).…”

Augmented Metacognition

“…Even if cognitions could be classified accurately [14], other information that is necessary to enable metacognition [12], such as task performance appraisals, may not be available to the technology [cf. 5,6,7]. Overcoming these issues may be necessary to truly enable metacognitive augmentation.…”