Analysis and comprehension of multimodal texts

Abstract

This research is part of an Australian Research Council linkage project that is developing a model of image-language relations in multimodal texts (Unsworth, Barnes & O'Donnell, 2006-2008. The materials used in the investigation were the 2005 and 2007 NSW Basic Skills Tests (BST). Data from the tests enabled an analysis of variance to compare the mean difficulty of BST items assessing the comprehension of different types of image-language relations which have been outlined by Unsworth and Chan (2008). As there was a range of difficulty for the items assessing each type of image-language relation, it was decided to investigate other aspects of the texts that might be causing difficulty. Accordingly, complexity within the targeted written language segments and complexity within the images were analysed separately using selected features of Functional Grammar (Halliday, 2004/ 1994) and Visual Grammar (Kress & van Leeuwen, 1996 to see whether these factors also affected item difficulty. In conclusion, the findings about multi-semiotic text complexities and item difficulties are considered in order to identify factors that might be relevant to the comprehension of multimodal texts and to suggest implications for future research about language and literacy learning in primary schools.

Multimodal Discourse Analysis of Classroom Oral Presentations at Tertiary Level

“…Focusing all discussed scenarios, last decade observed multimodal approach which has successfully marked a prominent impression on ELT (English Language Teaching) covering lesson materials, latest technologies, semiotization and resemiotization in academic discourses by using latest technologies (Kress, Jewitt, 2002b& 2008Kress, 2003;Jewitt, Ogborn & Tsatsarelis, 2001;Unsworth, 2001Unsworth, 2008bUnsworth & Chan, 2009;Daly & Unsworth, 2011). By modernizing the teaching-learning perspectives such approaches have also challenged the conventional teaching pedagogies (Schleppegrell, 2007).…”

An Introduction to Conversation Analysis

“…The assessment can only be understood as an initiating move when it is understood in relation to the hyperlinked material. The two modes work in complementarity (Daly & Unsworth, 2011) to construct a single meaning. Farina argues that where turns are constructed multimodally, recipients respond to the message as a single contribution and the modes are inseparable in terms of the construction of the turn.…”

“…In each of these the main element of the turn construction is the representation of embodied action with the text in the chat log functioning largely as a gloss of the embodiment. The turn is thus constructed from two modes that exist in a concurrent relationship (Daly & Unsworth, 2011); the meaning is equally available in each mode.…”

Carteles preventivos de la salud costarricense más allá de lo verbal: Un análisis multimodal

“…En cuanto a otras investigaciones desarrolladas que se relacionan con el objeto de estudio del presente artículo, estas están enfocadas, principalmente, en la lectura y la comprensión de textos multimodales y un segmento sustancial se ha desarrollado en los campos de la educación y la salud. Por ejemplo, en educación se ha estudiado la comprensión de las relaciones entre imágenes y palabras en textos multisemióticos 2 utilizando como punto de referencia la lingüística sistémico-funcional de Halliday (Daly y Unsworth, 2011). También se ha observado cómo se logra transferir significado por medio de diversos recursos semióticos con el objetivo de promover la alfabetización de contenidos científicos (Alvermann y Wilson, 2011).…”