Evidence and circularity in multimodal discourse analysis

Abstract

In the context of rapid theoretical development in multimodal discourse analysis, and of its growing interdisciplinary influence, it is crucial that those working in the field give due consideration to methodological rigour. The corpus-based approach described here offers a means of addressing some key methodological issues. Firstly, this approach provides a check on over- and under-interpretation and also reveals a more nuanced picture of data about specific genres than might be derived from even the closest observation of individual instances. Thus it helps avoid pitfalls associated with relying on hand-picked examples. Secondly, the semi-automated implementation of a multilayered annotation scheme, which separates the representation of layout from rhetorical structure, supports the empirical investigation of a variety of research questions, while minimizing the influence of the analyst on the data by delaying interpretation insofar as possible until it becomes unavoidable. This article illustrates the corpus-based approach through a contrastive case study of one very visual genre, product packaging, with data taken from two locales, Taiwan and the UK. In so doing, issues of the selection of texts for inclusion and corpus design are addressed and the principles and practicalities involved in data preparation are discussed. Consideration is also given to the types of question which such an approach enables us to explore. In addition, since the data analyzed here are drawn from different languages and cultures, the present study sheds light on some issues of interest from the perspective of localization. Finally, some benefits of the approach are suggested, among which not least is that a stronger basis for the critique of designs in turn supports identification of opportunities for their improvement. This is not possible when the analysis is itself circular.

Towards next-generation visual archives:image, film and discourse

“…Simplified and sharpened searchability in combination with improved usability and access is key to the successful integration of such automatic and manual annotation efforts. As pointed out in Thomas (2014), only large-scale corpus research where low-level annotation is automated, thereby postponing human interpretation for later phases of analysis, will be able to produce the unique brand of reliable and reproducible multimodal content or discourse analysis that the current paper has argued to be necessary.…”

“…These levels have been used to annotate several corpora of different kinds of multimodal documents (cf. Thomas 2009;Hiippala 2013Hiippala , 2014, allowing some significant issues in the methodology of visual communication research and its relation to empirical methods to be addressed (Thomas 2014). …”

An overview of research within the Genre and Multimodality framework

“…For a concrete benefit, XML has enabled the use of different programming languages for querying and visualizing GeM corpora, because they can work with XML data (Thomas 2007, Hiippala 2015c. Moreover, XML remains effective for representing structured data, which may be extended as necessary: Thomas (2014) shows how additional layers of description may be integrated into the GeM annotation schema to extend the model's capability to describe multimodal phenomena.…”

“…Yet another cross-cultural study is presented by Thomas (2014), who applies the GeM framework to study product packaging in the United Kingdom and Taiwan. Focusing on fast-moving consumer goods, such as toothpaste and shampoo packs, Thomas (2014) describes their genre structure and examines the differences between the two locales. His analysis builds on the more extensive work in Thomas (2009), which defines a set of common 'message types' in product packaging on the basis of a corpus of 24 packages.…”

Individual and Collaborative Semiotic Work in Document Design

“…They contrast the original version with their own redesign, which adopts the principles of information value zones proposed in Kress/van Leeuwen (2006). This transformation, however, is not motivated by communicative needs that drive the formation of genres, but guided by the exceedingly abstract concept of information value zones, which has drawn considerable criticism in research on information design and multimodality (Waller 2012, Thomas 2014.…”