How Representational Pictures Enhance Students’ Performance and Test-Taking Pleasure in Low-Stakes Assessment

Abstract

Abstract. Pictures are often used in standardized educational large-scale assessment (LSA), but their impact on test parameters has received little attention up until now. Even less is known about pictures’ affective effects on students in testing (i.e., test-taking pleasure and motivation). However, such knowledge is crucial for a focused application of multiple representations in LSA. Therefore, this study investigated how adding representational pictures (RPs) to text-based item stems affects (1) item difficulty and (2) students’ test-taking pleasure. An experimental study with N = 305 schoolchildren was conducted, using 48 manipulated parallel science items (text-only vs. text-picture) in a rotated multimatrix design to realize within-subject measures. Students’ general cognitive abilities, reading abilities, and background variables were assessed to consider potential interactions between RPs’ effects and students’ performance. Students also rated their item-solving pleasure for each item. Results from item-response theory (IRT) model comparisons showed that RPs only reduced item difficulty when pictures visualized information mandatory for solving the task, while RPs substantially enhanced students’ test-taking pleasure even when they visualized optional context information. Overall, our findings suggest that RPs have a positive cognitive and affective influence on students’ performance in LSA (i.e., multimedia effect in testing) and should be considered more frequently.

Challenges Solving Science Tasks with Text–Picture Combinations Persist beyond Secondary School

“…The absence of a positive effect of text-picture material compared to exclusively textual material is contrary to most previous studies, but the studies' material differed in meaningful ways. These differences may explain why the results of our study differ from that of previous studies (e.g., Hegarty & Just, 1993;Lindner et al, 2018;Saß et al, 2012). The way we created redundancy in our task material differed, for example, from studies of Saß et al (2012) and Lindner et al (2018).…”

“…These differences may explain why the results of our study differ from that of previous studies (e.g., Hegarty & Just, 1993;Lindner et al, 2018;Saß et al, 2012). The way we created redundancy in our task material differed, for example, from studies of Saß et al (2012) and Lindner et al (2018). Instead of repeating nearly all information units of the text in a depictive representation, we repeated task-relevant depictive information units in the text.…”

“…Several experimental studies revealed positive effects of redundancy in MERs. Additional redundant pictures in the stem of multiple-choice science items reduced item difficulty as compared to text-only items when the item stem presented information essential for solving items (d ¼ 0.53;Lindner et al, 2018). Saß et al (2012) also observed positive effects on students' test performance when combining redundant pictures in the stem, in the answer options, or in both with texts in multiple-choice items (g 2 ¼ .03 and .04 corresponding to ds of approximately 0.35 and 0.41).…”

“…In their low-powered study, Pociask and Morrison (2008) compared redundant and non-redundant MERs, but due to eliminating redundant features the amount of test material also differed between the groups. Saß et al (2012) and Lindner et al (2018) placed pictures into multiple-choice items and generated redundancy based on the text. This kind of redundant text-picture combinations is specific to assessment and does not reflect MERs in the scientific discourse; it therefore lacks ecological validity in this science-related context.…”

The Onset of Rapid-Guessing Behavior Over the Course of Testing Time: A Matter of Motivation and Cognitive Resources

“…As an indicator of students' current motivational level, we repeatedly measured students' task enjoyment while working on the items. We did so with a one-item measure (see Lindner et al, 2016), asking students how much fun they had solving the current item (i.e., “Working on this item was fun for me”). We assumed that lower enjoyment ratings would indicate lower motivational resources.…”