Visualizing digital media interactions

Abstract

Instrumental music performance is a well-established case of real-time interaction with technology and, when extended to ensembles, of interaction with others. However, these interactions are fleeting and the opportunities to reflect on action is limited, even though audio and video recording has recently provided important opportunities in this regard. In this paper we report on research to further extend these reflective opportunities through the capture and visualization of gestural data collected during collaborative virtual performances; specifically using the digital media instrument Jam2jam AV and the specifically-developed visualization software Jam2jam AV Visualize. We discusses how such visualization may assist performance development and understanding. The discussion engages with issues of representation, authenticity of virtual experiences, intersubjectivity and wordless collaboration, and creativity support. Two usage scenarios are described showing that collaborative intent is evident in the data visualizations more clearly than in audio-visual recordings alone, indicating that the visualization of performance gestures can be an efficient way of identifying deliberate and co-operative performance behaviours.

Reviewing reflection

“…First, of the papers we reviewed, 15 dealt with education in some way [14,15,16,17,30,43,48,50,51,53,70,77,78,79,90,93,98]. Specific foci ranged from reflection as a means of educating working professionals [16] to music [14,43] or dance [53] instruction, to teaching programming [90].…”

“…Specific foci ranged from reflection as a means of educating working professionals [16] to music [14,43] or dance [53] instruction, to teaching programming [90].…”

“…Furthermore, Schön's [76] work on the reflective practitioner is often cited in professional education. Indeed, of the five education-oriented papers that cited a definition of reflection, three [14,51,90] cite Schön [76] and two [53,90] cite Dewey [20], while others [79] cite Moon [53] and Norman [71].…”