People sense the world by exploiting correlations between their physical actions and the changing sensory input that results from those actions. Interfaces that translate non-human sensor data to signals that are compatible with the human senses can therefore augment our abilities to make sense of the world. This insight has recently sparked an increase in projects that explore sensemaking and the creation of novel human experiences across scientific and artistic disciplines. However, there currently exists no constructive dialogue between artists and scientists that conduct research on this topic. In this position paper, we identify the theory and practice of sensory augmentation as a domain that could benefit from such a dialogue. We argue that artistic and scientific methods can complement each other within research on sensory augmentation and identify six thematic starting points for a dialogue between the arts and sciences. We conducted a case study to explore these conjectures, in which we instigated such a dialogue on a small scale. The case study revealed that the six themes we identified as relevant for a dialogue on sensory augmentation emerge rather spontaneously in such a dialogue and that such an exchange may facilitate progress on questions that are central to the theory and practice of sensory augmentation. Overall, this position paper contributes preliminary evidence for the potential of, and a starting point for, a dialogue between the arts and sciences that advances our understanding of sensory augmentation and the development of applications that involve it.

No abstract

Deaf and hearing people can encounter challenges when communicating with one another in everyday situations. Although problems in verbal communication are often seen as the main cause, such challenges may also result from sensory differences between deaf and hearing people and their impact on individual understandings of the world. That is, challenges arising from a sensory gap. Proposals for innovative communication technologies to address this have been met with criticism by the deaf community. They are mostly designed to enhance deaf people's understanding of the verbal cues that hearing people rely on, but omit many critical sensory signals that deaf people rely on to understand (others in) their environment and to which hearing people are not tuned to. In this perspective paper, sensory augmentation, i.e., technologically extending people's sensory capabilities, is put forward as a way to bridge this sensory gap: (1) by tuning to the signals deaf people rely on more strongly but are commonly missed by hearing people, and vice versa, and (2) by sensory augmentations that enable deaf and hearing people to sense signals that neither person is able to normally sense. Usability and user-acceptance challenges, however, lie ahead of realizing the alleged potential of sensory augmentation for bridging the sensory gap between deaf and hearing people. Addressing these requires a novel approach to how such technologies are designed. We contend this requires a situated design approach.