A Relational Reasoning Approach to Text-Graphic Processing

Abstract

We propose that research on text-graphic processing could be strengthened by the inclusion of relational reasoning perspectives. We briefly outline four aspects of relational reasoning: analogies, anomalies, antinomies, and antitheses. Next, we illustrate how textgraphic researchers have been conducting research aligned with aspects of relational reasoning, although not deliberately. We call for future research on intentionally designed textgraphic pairings that should be empirically tested for their ability to support relational reasoning. Finally, we argue that relational reasoning may help explain some of the mixed results and unintended outcomes that often appear in the text-graphic research literature.Researchers have developed rich models to account for both text processing (Kintsch 1992;McNamara et al. 1996) and graphic processing (Ainsworth 2006; Ainsworth and Burcham 2007; Hegarty 2011) in isolation. Yet, when text and graphics are combined, learners tend to infer very different meanings than when these representations are presented independently (Hagan 2007). Often, the combination of a text with a graphic is more than just the sum of the two parts. This is because when combined, text and graphics can communicate relationships among or between the informational representations (Hegarty 2011;McCabe and Castel 2008). In other words, text and graphics, when presented together, are inherently relational.In this article, we briefly review the extant text-graphic research as it relates to four aspects of relational reasoning: analogies, anomalies, antinomies, and antitheses. Next, we present a number of examples that draw upon these four aspects of relational reasoning as they relate to learning with texts and graphics. We go on to suggest that viewing text-graphic processing as an exercise in relational reasoning may help explain a number of inconsistent results that have historically hampered research in this area. Finally, we conclude with Educ Psychol Rev

How affective charge and text–picture connectedness moderate the impact of decorative pictures on multimedia learning.

“…Revealed impacts on learning outcomes might serve as a first indicator for their validity. However, more detailed measures (e.g., eye-tracking) should be conducted in order to provide deeper insights into learners’ relational reasoning (Danielson & Sinatra, 2016). Besides cognition, educational researchers stress that emotions are inherently motivational, that is that emotions might exert powerful motivational influences (e.g., Pekrun, 2006; Plass & Kaplan, 2015).…”

“…Recent evidence suggests that learners always actively construct relations between a learning text and accompanied pictures (Danielson et al, 2015; Danielson & Sinatra, 2016) to build integrated mental models (Schnotz, 2014), irrespective of the pictures’ relevance for the learning goal. To what extent such a relation is established might be strongly influenced by two facets of multimedia design.…”

Multidimensional Engagement in Learning—an Integrated Instructional Design Approach

“…They establish multiperspective relationships among elements of knowledge acquisition and by standard-based knowledge revision (e.g., McMillan, 2013). As reasoning can be stimulated by integrating analogies (two domains are interpreted in terms of one another), anomalies (something that does not something is by ascertaining what it is not), and antitheses (with elements in oppositional relations to one another) into learning experiences (Danielson & Sinatra, 2017). As a second general instructional strategy, standards (goals, norms, guidelines etc.)…”

When words and graphs move the eyes:The processing of multimodal causal relations

“…For Zacks and Tversky (1999), statistical graphs are cognitive artefacts used in scientific discourse both to reason and to communicate data, as they help interpret the results obtained in scientific research and allow communication and dissemination of results and conclusions. Statistical graphs are of paramount importance in the development and communication of scientific research, which has led to conceptualization of them as powerful objects (Kosslyn, 1994;Cheng & Simon, 1995;Tufte, 2001;Danielson & Sinatra, 2016). Furthermore, the visual representation of data facilitates the interpretation and comprehension of numerical or statistical values associated with variables under study.…”

“…Multisemiotic written texts, composed of multiple semiotic systems (e.g., words, graphs, images, diagrams), are widely employed in academic and professional contexts and especially in cross-disciplinary genres (Parodi, 2010(Parodi, , 2015. Graph/word texts, for example, have been identified as fundamental written discourse tools in the construction and transmission of specialized knowledge (Kosslyn 1994;Schnotz & Baadte, 2015;Danielson & Sinatra, 2016;Parodi & Julio, 2016. Similarly, economics reports are written using a wide range of diverse semiotic systems, such as verbal and diagrammatic (Parodi, Julio, & Vásquez-Rocca, 2015).…”