Reading Multimodal Texts: Perceptual, Structural and Ideological Perspectives

Abstract

This article presents a tripartite framework for analyzing multimodal texts. The three analytical perspectives presented include: (1) perceptual, (2) structural, and (3) ideological analytical processes. Using Anthony Browne's picturebook Piggybook as an example, assertions are made regarding what each analytical perspective brings to the interpretation of multimodal texts and how these perspectives expand readers' interpretive repertoires. Drawing on diverse fields of inquiry, including semiotics, art theory, visual grammar, communication studies, media literacy, visual literacy and literary theory, the article suggests an expansion of the strategies and analytical perspectives readers being to multimodal texts and visual images. Each perspective is presented as necessary but insufficient in and of itself to provide the necessary foundation for comprehending texts. It is through an expansion of the interpretive strategies and perspectives that readers bring to a multimodal text, focusing on visual, textual, and design elements that readers will become more proficient in their interpretive processes.

Developing Students' Interpretive Repertoires

“…ways in which students construct meaning with multimodal texts, researchers have begun to expand their vision of what it means to be literate and to endorse the significant role that visual literacy plays in contemporary classrooms (Arizpe & Styles, 2003;Callow, 1999;Pantaleo, 2008;Serafini, 2010).…”

“…Readers oscillate back and forth among the sign systems during the reading of picturebooks and, "…the total effect depends not only on the unison of the text and illustrations, but also on the perceived interaction or transaction between the two" (Sipe, 1998, p. 98). As readers encounter visual images and graphic design elements in addition to the written language presented in contemporary picturebooks, the repertoire of strategies students need to draw upon must expand to deal with the aspects of these texts beyond decoding of written language (Pantaleo, 1995(Pantaleo, , 2004bSerafini, 2010).…”

Visual Literacy From the Perspective of the VTS Method

“…They highlight the importance of the teachers' own ability to use and understand precise meta-language in different socio cultural contexts. When this is mastered, the teacher is able to teach and facilitate visual literacy (Kern, 2000;Serafini, 2010;Unsworth, 2006.) The development of the VTS method itself, however, has taken as its starting point that no previous knowledge of art is required of the teacher.…”

“…According to some researchers, postmodern picture books can also be used in the teaching of visual literacy. Teaching then directs attention to the use of, for example, framing, lines and colours in the picture book as well as the different proportions of the text (see for example O'Neil, 2011;Serafini, 2010;Serafini & Gee, 2014;Stafford, 2011;Unsworth, 2006). In addition to picture books, teaching can also make use of graphic novels (Wolfen & Kleijwegtin, 2012).…”

“…Nowadays the term multimodal is applied to texts which make use not only of image and text but also, for example, of graphic elements, hypertexts, links and the moving image. In addition to visual and verbal, we can also talk about auditive, kinetic and numerical forms of representation, which are called modes or modalities (See for example Cope & Kalantzis, 2000;Kress, 2003;Kress & Leeuwen, 2006;New London Group, 1996;Räsänen, 2015;Serafini, 2010;Unsworth, 2001;2006). It would thus seem that visual material deriving from particularly diverse starting points is suitable for training visual literacy.…”