Investigating the Split‐Attention Effect in Computer‐Based Assessment: Spatial Integration and Interactive Signaling Approaches

Abstract

Many test items use both an image and text, but present them in a spatially separate manner. This format could potentially cause a split-attention effect in which the test taker's cognitive load is increased by having to split attention between the image and text, while mentally integrating the two sources of information. We investigated the split-attention effect in computer-based assessment by implementing: (a) spatial integration which embeds text information into a diagram as labels, and (b) interactive signaling which highlights a segment of a diagram when test takers hover their mouse over an answer option that refers to the relevant segment. Adult participants solved computer-based geometry items in which spatial integration was used as a within-subject variable and interactive signaling was used as a between-subject variable. The main findings showed that, compared with the nonintegrated and nonsignaled items, (a) spatial integration significantly increased item-solving efficiency indicated by reduced time on task, especially for test takers who had higher prior knowledge, and (b) interactive signaling increased item-solving efficiency only in the spatially integrated items. The current findings suggest that applying multimedia design principles for reducing split-attention to test item design could potentially reduce test takers' construct-irrelevant cognitive load in computer-based assessment.

Dynamic signals in instructional videos support students to navigate through complex representations: An eye‐tracking study

“…Furthermore, Jeung et al (1997) reported that combined audio and visual instruction was only effective for learning if dynamic signals were present in the case of complex geometry problems as compared to problems with low-visual search demands. Likewise, a study by Moon et al (2022) found that interactive signaling (i.e., on demand highlights) in complex geometry material improved performance only when the diagrams were already optimized in terms of spatial integration. From these studies, it can be tentatively concluded that the effectiveness of signals in learning depends on the complexity of the visual representation and the type of signaling that is implemented in instructional videos and similar multimedia materials.…”

Process Data in Computer-Based Assessment

“…In a similar vein, process data have been applied in experimental research settings to better understand the effects that changing test item characteristics has on test-taker cognition, such as varying item design factors (e.g., Arslan et al, 2020; Ehrhart & Lindner, 2023; Lindner, Eitel, et al, 2017; Lindner, Lüdtke, et al, 2017; Moon et al, 2019, 2022; Saß et al, 2017, Strobel et al, 2018). Process data have also been used to gain insights into test-taker behavior in a broader sense (e.g., Deribo et al, 2021; Goldhammer et al, 2014; Lindner et al, 2014; Maddox et al, 2018; Ranger et al, 2021; Wise, 2017).…”

“…Especially in the area of educational testing, we see increasing research efforts in order to complement traditional test scores in (large-scale) assessments by process-related inferences, for example, to consider test-taker’s response behavior (e.g., Araneda et al, 2022; Arslan et al, 2021; Gao et al, 2022; Wise, 2017), or better understand problem-solving approaches and response strategies (e.g., Arslan et al, 2020; Gong et al, 2023; Greiff et al, 2015; He et al, 2016, 2021, 2023; He & von Davier, 2016; Jiang et al, 2023; Moon et al, 2022). This research area has evolved considerably in the past years when the testing mode in many prominent large-scale assessments, such as PISA (Programme for International Student Assessment), TIMSS (Trends in International Mathematics and Science Study), and PIAAC (Programme for the International Assessment of Adult Competencies) progressively shifted from paper-pencil to computer-based formats (see, e.g., Buerger et al, 2016; Meinck et al, 2017) and made the data available for research.…”

Learning from screencast software tutorials: A comparison of cognitive load in dual and single‐monitor learning environments

“…The central hypothesis of split attention research is that cognitive load increases when learners must mentally integrate information from two separate sources, thus reducing working memory resources available for germane processing and, ultimately, learning (Paas et al, 2003). There is substantial evidence in support of this effect for different types of instructional materials, including multimedia, across a wide range of domains, including software training and multimedia learning environments (e.g., Sweller et al, 1990; Sweller & Chandler, 1994; Chandler & Sweller, 1996; Cerpa, Chandler & Sweller, 1996; Yeung et al, 1998; Mayer et al, 1995; Owens & Sweller, 2008; Cierniak et al, 2009; Mutlu‐Bayraktar et al, 2022; Moon et al, 2022). Furthermore, Ayres and Sweller (2014) highlight that the effect is most pronounced for novices attempting to integrate high element‐interactivity information with multiple interacting cognitive, psychomotor, and verbal elements that must be processed simultaneously.…”