The Power of Sound Design in a Moving Picture: an Empirical Study with emoTouch for iPad

Abstract

The art of sound design for a moving picture rests basically on the work experience of pragmatists. This study tries to establish some guidelines on sound design: In an experiment 240 participants gave feedback about their emotions while watching two videos, each combined with four different audio tracks – music, sound effects, full sound design (music and sound effects) and no audio (as the comparative "null" version). Each participant viewed an audiovisual combination once to prevent habituation. The lead author employed a tablet-computer with the emoTouch-application serving as a mapping tool to provide information about the emotional responses. The participants moved a marker on the tablet's touch screen in a two-dimensional rating scale describing their felt immersion and suspense. A 3-factor-ANOVA showed significant increases of the median (and maximum) values of immersion and suspense when the participants listened to music and/ or sound effects. These values were always compared to the induced emotions of the participants who watched the videos with no audio at all. The video with full sound design audio tracks increased the median immersion values up to four times and the median suspense values up to 1.4 times. The median suspense values of the video with either music or sound effects dropped by 40 percent compared to the median suspense values of the null version. In contrast, the median immersion values were increased up to 3.6 times. The findings point to the importance of sound effects in an appropriate mix with music to enhance the viewers induced immersion and suspense.

Emotionally congruent music and text increase immersion and appraisal

“…Thinking laterally, interesting analogies can be found in film music research. Studies revealed that music and sound effects influence emotions experienced by the viewer [ 74 ] and enhance immersion and suspense [ 75 ]. Film music furthermore affects the semantic evaluation of film scenes [ 76 ], it can foster empathy with the characters [ 77 ], and the content of film excerpts is remembered more easily when music and pictures convey a congruent mood [ 78 ].…”

More horror due to specific music placement? Effects of film music on psychophysiological responses to a horror film

“…Aside from music, sound effects also have an influence on felt emotions and immersion while watching a movie. Kock and Louven (2018) combined silent film scenes with sound effects, music, or sound design (music + sound effects). Two pieces of Modest Mussorgsky's "Pictures at an Exhibition" in the original piano version were used as musical accompaniment.…”