Multimodal transcription as academic practice: a social semiotic perspective

Abstract

With the increasing use of video recording in social research methodological questions about multimodal transcription are more timely than ever before. How do researchers transcribe gesture, for instance, or gaze, and how can they show to readers of their transcripts how such modes operate in social interaction alongside speech? Should researchers bother transcribing these modes of communication at all? How do they define a 'good' transcript? In this paper we begin to develop a social semiotic framework to account for transcripts as artefacts, treating them as empirical material through which transcription as a social, meaning making practice can be reconstructed. We look at some multimodal transcripts produced in conversation analysis, discourse analysis, social semiotics and microethnography, drawing attention to the meaning-making principles applied by the transcribers. We argue that there are significant representational differences between multimodal transcripts, reflecting differences in the professional practices and the rhetorical and analytical purposes of their makers. IntroductionTranscription is a common academic practice. In social research investigating language and communication, a 'transcript' usually refers to a distinctive genre associated with turning a strip of 'naturally' occurring talk -for example a job interview, a conversation at the dinner table -into writing. This genre has analytical as well as rhetorical purposes: to develop insights into the moment-by-moment and in situ construction of social reality and to provide evidence in developing an argument for an academic audience. With the increasing use of video recording in social research, methodological questions about multimodal transcription are more timely than ever before. How do researchers transcribe gesture, for instance, or gaze, and how can they show to readers of their transcripts how such modes operate alongside speech? Should researchers bother to transcribing these modes of communication at all? What are the epistemological implications of choices of inclusion and exclusion? What does one gain from inclusion of modes other than speech if the aim of transcription is to focus on a selection of the vast amount of data collected? In this paper we investigate how some researchers, including ourselves, have dealt with these issues. We discuss the emergence of multi-

Video in Sociomaterial Investigations

“…Therefore, and as you will see throughout this article, there is an attempt to preserve features of the extract in sufficient detail for others to be able to assess the analytic claims and observations being made, but these remove a great deal of the detail included in analytic transcripts and are tailored for the character of the episode under scrutiny. Nevertheless textual renderings of video are inevitably deficient in many ways (Bezemer and Mavers, 2011 ,,,,,,,,,,,,,,,,,,,,,,,,, D: there: we go.-------------°we'll° just take your bib P: Head right___________down____right,,,,,,,,,,,,,,,,,,,,,, Arms armrest_________up,,down,,,,,,,,,,,,,,armrest______ …”

Using relationships as a tool: early childhood educators’ perspectives of the child–caregiver relationship in a childcare setting

“…The videos were transcribed in written form by the first author, following the written transcription of video data method described by Bezemer and Mavers (2011). This involved selecting and highlighting frames that related to the research question then detailing in writing all interaction between the ECE and the children (including transcription of vocalisations, verbalisations, physical touch and eye gaze between ECE and child, as well as interactions by both ECE and child with other children and staff members).…”

Unpacking physics representations: Towards an appreciation of disciplinary affordance

“…These excerpts present the sequential development of how the students connected their circuit. The video data will be presented in a form of "multimodal transcription" [68,69]] that involves the verbatim transcriptions and the situational illustrations of reconstructions of the laboratory setup.…”