Intersemiotic Translation as Resemiotisation: A Multimodal Perspective

O’Halloran, Kay; Tan, Sabine; Wignell, Peter

doi:10.4000/signata.1223

Cited by 37 publications

(21 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(Bezemer & Jewitt, 2018). It can be tested and implemented in the process of resemiotisation also known as intersemiotic translation when meanings are translated from the target language into another non-verbal system such as images of symbols (O'Halloran, Tan, & Wignell, 2016). Intersemiotic translation can be investigated through the meta-functionally organised systems of meaning, following Halliday's systematic functional theory 12 and multimodal discourse analysis (SF-MDA).…”

Section: Multimodality In Language Educationmentioning

confidence: 99%

“…The interpersonal function in multimodal texts with the visual content includes the interactive relationship through the system of conduct (viewer's participation), the system of social distance (size and frame), the system of subjectivity (involvement or detachment, power of the depicted figures) and objectivity (frontal or top-down images). The textual function constitutes the organisation of experiential, logical and interpersonal meanings into messages (Kress & van Leeuwen, 2006;Painter, Martin, & Unsworth, 2014;O'Halloran, Tan, & Wignell, 2016;Mills & Unsworth, 2016, Although the terminology used by scholars for the three meta-functions may vary to a certain degree, they have always been recognised in the area of MDA. Namely, Kress and van Leeuwen (2006) referred to the experiential meaning as representational meaning.…”

Section: Research Questionsmentioning

confidence: 99%

See 1 more Smart Citation

Energy Of Visual And Verbal Modalities In Language Education

Stec¹

2019

The European Proceedings of Social and Behavioural Sciences

View full text Add to dashboard Cite

Pictures, like words are omnipresent in our lives. Each form of communication carries visible (clear) and invisible (hidden) messages. This paper will describe the research conducted during a workshop about visual and verbal input in language education. The workshop was addressed to students and teachers who participate in children's language education. A focus was on the sociocultural context of learning and visual literacy as essential skills for reading multimodal texts and transferring information in the 21 st century. There were two questions stated: What is the role of verbal and visual modalities in language education?What is the image-text relationship in transferring information? The qualitative, sociocultural and MDA approaches were applied to raise participant's awareness of image-text intermodality. The idea was also to practise selection and evaluation of ELT materials. The paper hopes to increase the role of visual methodology and multimodal perspective in language education.

show abstract

Section: Multimodality In Language Educationmentioning

confidence: 99%

Section: Research Questionsmentioning

confidence: 99%

Energy Of Visual And Verbal Modalities In Language Education

Stec¹

2019

The European Proceedings of Social and Behavioural Sciences

View full text Add to dashboard Cite

show abstract

“…An important point needs to be made regarding the translational dimension. In a landmark paper on translation, Jakobson (1959: 233) defines "three ways of interpreting (translating) a verbal sign": It is the third "way" that is most relevant to the present concern: "intersemiotic translation" -or "resemiotisation" (Eco 1979, O'Halloran et al 2016. This consists in using a different medium to make the source text or message accessible.…”

Section: Trans-semiotic Adaptationmentioning

confidence: 99%

Breaking Virginia’s Waves (1931): from page to stage

Lapaire¹

2019

miranda

View full text Add to dashboard Cite

show abstract

“…The approach has been adapted to the analysis of images (Kress and Van Leeuwen, 2001), art and architecture (O’Toole, 2011), music (Van Leeuwen, 1999) and mathematical images and symbolism (O’Halloran, 2008) in artefacts such as websites and infographics (O’Halloran et al, 2016) and films (Bateman, 2014; Bateman and Schmidt, 2012), among others. SF-MDA is informed by a number of defining principles from SFT (O’Halloran et al, 2016). The two most relevant here are: (1) the metafunctional hypothesis; and (2) resemiotization.…”

Section: Systemic Functional Multimodal Discourse Analysis (Sf-mda)mentioning

confidence: 99%

A multimodal approach to visual thinking: the scientific sketchnote

Fontecha

O’Halloran²,

Tan³

et al. 2018

Visual Communication

Self Cite

View full text Add to dashboard Cite

There is a growing interest in the use of visual thinking techniques for promoting conceptual thinking in problem solving tasks as well as for reducing the complexity of ideas expressed in scientific and technical formats. The products of visual thinking, such as sketchnotes, graphics and diagrams, consist of ‘multimodal complexes’ that combine language, images, mathematical symbolism and various other semiotic resources. This article adopts a social semiotic perspective, more specifically a Systemic Functional Multimodal Discourse Analysis approach, to study the underlying semiotic mechanisms through which visual thinking makes complex scientific content accessible. To illustrate the approach, the authors analyse the roles of language, images, and mathematical graphs and symbolism in four sketchnotes based on scientific literature in physics. The analysis reveals that through the process of resemiotization, where meanings are transformed from one semiotic system to another, the abstractness of specialized discourses such as physics and mathematics is reduced by multimodal strategies which include reformulating the content in terms of entities which participate in observable (i.e. tangible) processes and enhancing the reader/viewer’s engagement with the text. Moreover, the compositional arrangement creates clear stages in the development of the ideas and arguments that are presented. In this regard, visual thinking is a form of cultural communication through which abstract ideas are translated and explained using a multimodal outline or summary of essential parts by adapting resources (e.g. linguistic resources and mathematical graphs), using new resources (e.g. stick figures and other simple schematic drawings) and maintaining others from the original text (e.g. mathematical symbolic notation), resulting in a congruent (or concrete) depiction of abstract concepts and ideas for a non-specialist audience.

show abstract

Intersemiotic Translation as Resemiotisation: A Multimodal Perspective

Cited by 37 publications

References 11 publications

Energy Of Visual And Verbal Modalities In Language Education

Energy Of Visual And Verbal Modalities In Language Education

Breaking Virginia’s Waves (1931): from page to stage

A multimodal approach to visual thinking: the scientific sketchnote

Contact Info

Product

Resources

About