Effects of Reducing the Cognitive Load of Mathematics Test Items on Student Performance

Gillmor, Susan C.; Poggio, John P.; Embretson, Susan E.

doi:10.5038/1936-4660.8.1.4

Cited by 20 publications

(14 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similarly, on an advanced web applications course for graduate students, CLT was used to develop an online programming tool, and researchers found that students performed best when they were able to view examples of code during the learning of new material (Heo and Chow, 2005). When CLT was applied to teaching math to middle school students, researchers found that student performance was improved by signaling important information, improving the aesthetic of item organization, and removing extraneous content (Gillmor et al ., 2015). Previous studies on teaching programming to novice learners have also found that using CLT led to better learning as well as an increase in self-efficacy and reduction in the perception of difficulty (Mason and Cooper, 2013).…”

Section: Discussionmentioning

confidence: 99%

Successful Integration of Data Science in Undergraduate Biostatistics Courses Using Cognitive Load Theory

Guzman

Pennell

Nikelski

et al. 2019

LSE

View full text Add to dashboard Cite

Biostatistics courses are integral to many undergraduate biology programs. Such courses have often been taught using point-and-click software, but these programs are now seldom used by researchers or professional biologists. Instead, biology professionals typically use programming languages, such as R, which are better suited to analyzing complex data sets. However, teaching biostatistics and programming simultaneously has the potential to overload the students and hinder their learning. We sought to mitigate this overload by using cognitive load theory (CLT) to develop assignments for two biostatistics courses. We evaluated the effectiveness of these assignments by comparing student cohorts who were taught R using these assignments (n = 146) with those who were taught R through example scripts or were instructed on a point-and-click software program (control, n = 181). We surveyed all cohorts and analyzed statistical and programming ability through students’ lab reports or final exams. Students who learned R through our assignments rated their programming ability higher and were more likely to put the usage of R as a skill in their curricula vitae. We also found that the treatment students were more motivated, less frustrated, and less stressed when using R. These results suggest that we can use CLT to teach challenging material.

show abstract

Section: Discussionmentioning

confidence: 99%

Successful Integration of Data Science in Undergraduate Biostatistics Courses Using Cognitive Load Theory

Guzman

Pennell

Nikelski

et al. 2019

LSE

View full text Add to dashboard Cite

show abstract

“…The spatial integration approach was expected to reduce cognitive effort by reducing the amount of mental effort needed to integrate information from the legend and the diagram (e.g., Moreno & Mayer, 1999; Sweller & Chandler, 1994; Tarmizi & Sweller, 1988). The interactive signaling approach was expected to reduce extraneous load by removing the need to search for corresponding information in the diagram and the answer options (e.g., Gillmor et al., 2015; Lin & Atkinson, 2011; Richter et al., 2016; Schneider et al., 2018). Therefore, we hypothesized that each of the integration aids (i.e., spatial integration and interactive signaling) would enhance item‐solving efficiency (i.e., reduce time on task) as compared to a baseline condition without integration aids (see Figure 1a for an example baseline condition).…”

Section: Methodsmentioning

confidence: 99%

“…Computer‐based testing provides new opportunities for enhanced item design (e.g., several new item formats, such as drag‐and‐drop or inline choice) but may also be more demanding than paper‐based testing because of the extra effort that test takers need to invest to interact with digital items (Carpenter & Alloway, 2018; Clariana & Wallace, 2002). All of these item design related aspects could add extraneous cognitive load (Arslan et al., 2020; Cayton‐Hodges et al., 2012; Dolan et al., 2011; Moon et al., 2019; Thomas, 2016), which has the potential to interfere with test takers’ ability to demonstrate their skills and knowledge in the target constructs (Elliott et al., 2009; Parker et al., 2013), and thus impair the validity of test score interpretations (Gillmor et al., 2015).…”

Section: Introductionmentioning

confidence: 99%

“…However, effects of signaling in the testing context have not yet been extensively studied. While a limited number of existing studies (Gillmor et al., 2015; Jarodzka et al., 2015; Saß et al., 2017) have provided preliminary evidence for positive effects of signaling on test performance outcomes, they only examined noninteractive signaling techniques such as bolded text (Gillmor et al., 2015; Saß et al., 2017) or added referential connections, such as arrows (Saß et al., 2017) and combined different types of signaling techniques at once. There remains an unanswered question as to whether interactive technology‐enhanced signaling techniques also benefit test takers’ item processing.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Investigating the Split‐Attention Effect in Computer‐Based Assessment: Spatial Integration and Interactive Signaling Approaches

Moon

Lindner

Arslan

et al. 2022

Educational Measurement

View full text Add to dashboard Cite

Many test items use both an image and text, but present them in a spatially separate manner. This format could potentially cause a split-attention effect in which the test taker's cognitive load is increased by having to split attention between the image and text, while mentally integrating the two sources of information. We investigated the split-attention effect in computer-based assessment by implementing: (a) spatial integration which embeds text information into a diagram as labels, and (b) interactive signaling which highlights a segment of a diagram when test takers hover their mouse over an answer option that refers to the relevant segment. Adult participants solved computer-based geometry items in which spatial integration was used as a within-subject variable and interactive signaling was used as a between-subject variable. The main findings showed that, compared with the nonintegrated and nonsignaled items, (a) spatial integration significantly increased item-solving efficiency indicated by reduced time on task, especially for test takers who had higher prior knowledge, and (b) interactive signaling increased item-solving efficiency only in the spatially integrated items. The current findings suggest that applying multimedia design principles for reducing split-attention to test item design could potentially reduce test takers' construct-irrelevant cognitive load in computer-based assessment.

show abstract

“…For example, when training students to identify important features in a given situation, a multiple choice format requires students to switch back and forth between the choices and the figure, incurring much cognitive load. Previous research on problem format (Chen and Chen 2009; Gillmor et al 2015; Huang et al 2015) showed that students perceive higher cognitive load completing traditional multiple choice questions.…”

Section: Experiments 1: Developing Physics Expertise Using Deliberate mentioning

confidence: 97%

Researching for better instructional methods using AB experiments in MOOCs: results and challenges

Chen

Chudzicki

Palumbo

et al. 2016

RPTEL

View full text Add to dashboard Cite

We conducted two AB experiments (treatment vs. control) in a massive open online course. The first experiment evaluates deliberate practice activities (DPAs) for developing problem solving expertise as measured by traditional physics problems. We find that a more interactive drag-and-drop format of DPA generates quicker learning than a multiple choice format but DPAs do not improve performance on solving traditional physics problems more than normal homework practice. The second experiment shows that a different video shooting setting can improve the fluency of the instructor which in turn improves the engagement of the students although it has no significant impact on the learning outcomes. These two cases demonstrate the potential of MOOC AB experiments as an open-ended research tool but also reveal limitations. We discuss the three most important challenges: wide student distribution, "open-book" nature of assessments, and large quantity and variety of data. We suggest possible methods to cope with those. General backgroundWith tens of thousands of registered students, massive open online courses (MOOCs) hold great promise for data-driven education research Reich 2015;Williams 2013). Research in MOOC started almost simultaneously with the launch of the first MOOCs, covering topics from student behavior (Han et al. 2013; Kizilcec et al. 2013, edX;Koedinger et al. 2015) and instructional strategies Kim et al. 2014;Piech et al. 2013) to learning outcomes (Colvin et al. 2014). An important recent advance in MOOC-based education research is the capability to conduct an "AB experiment" on major MOOC platforms (Anderson et al. 2014; edX, n.d.;Williams and Williams 2013).Also called split testing, AB testing, or randomized controlled trial, AB experiments are a research method that is used by a wide range of disciplines ranging from marketing and business intelligence to medical research and social sciences (Chalmers et al. 1981;Kohavi and Longbotham 2015). Simply put, in a MOOC AB experiment, different instructional materials are presented to different groups of students, and the effectiveness of the materials can be evaluated by measuring the behavior and learning outcomes of students using various metrics. Chen -016-0034-4 MOOC AB experiments can be a powerful tool to help sort through the enormous variety of instructional choices. Koedinger et al. (2013) estimated the raw number of possible instructional choices to be on the order of an impossible 3 30 , which includes decisions such as more worked examples vs. more tests, immediate vs. delayed feedback, and longer vs. shorter videos. With large number of students and automated group randomization scheme, multiple MOOC AB experiments can be easily implemented in a single course, enabling researchers to accumulate evidence on different instructional designs at a much faster rate. However, the intrinsic "openness" of MOOCs also poses unique challenges on both the design of AB experiments and the analysis of the result. For example, unlike most clinical ...

show abstract

Effects of Reducing the Cognitive Load of Mathematics Test Items on Student Performance

Cited by 20 publications

References 31 publications

Successful Integration of Data Science in Undergraduate Biostatistics Courses Using Cognitive Load Theory

Successful Integration of Data Science in Undergraduate Biostatistics Courses Using Cognitive Load Theory

Investigating the Split‐Attention Effect in Computer‐Based Assessment: Spatial Integration and Interactive Signaling Approaches

Researching for better instructional methods using AB experiments in MOOCs: results and challenges

Contact Info

Product

Resources

About