How Can the Use of Data from Computer-Delivered Assessments Improve the Measurement of Twenty-First Century Skills?

Ramalingam, Dara; Adams, Raymond J.

doi:10.1007/978-3-319-65368-6_13

Cited by 10 publications

(11 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nevertheless, most of the inaccurate readers were also slow readers and were, therefore, identified by the reading fluency criteria in GBA. It could also be argued that the skill assessments with our GBA provide more information than the assessments with the traditional PPT (Greiff et al, 2015;Ramalingam & Adams, 2018). Therefore, although the grade-normative data from GBA was less sensitive when compared with PPT data, it could be that our GBA was more sensitive in finding subtle differences in students' skills, and this kind of information could be useful for picking out students who seemingly are typically developing readers but who might in fact be borderline cases and might benefit from some extra attention and instruction to improve their reading skills further.…”

Section: Discussionmentioning

confidence: 99%

“…Recently, CBA has been designed to include dynamic assessment for the use of computerized adaptive testing (Navarro et al, 2018), which has already shown to work among primary school children in mathematical skills (Martin & Lazendic, 2018). Furthermore, such CBA shows potential for using log files to investigate learning process and gaining detailed data to increase the validity and reliability of the assessment (Ramalingam & Adams, 2018) and in interpreting the individual responses (Greiff, Wu¨stenberg, & Avvisati, 2015). If a CBA is designed so that it functions as an integrated part of a digital game, assessments in a game environment can keep young students more engaged and motivated than traditional assessments encouraging students to demonstrate their full potential (e.g., Perrotta, Featherstone, Aston, & Houghton, 2013).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Identification of Reading Difficulties by a Digital Game-Based Assessment Technology

Hautala

Heikkilä

Nieminen

et al. 2020

Journal of Educational Computing Research

View full text Add to dashboard Cite

Computerized game-based assessment (GBA) system for screening reading difficulties may provide substantial time and cost benefits over traditional paper-and-pencil assessment while providing means also to individually adapt learning content in educational games. To study the reliability and validity of a GBA system to identify struggling readers performing below a standard deviation from mean in paper-and-pencil test either in raw scores and grade-normative scores, a large-scale study with first to fourth grade students ( N = 723) was conducted, where GBA was administrated as a group test by tablet devices. Overall, the results indicated that the GBA can be successfully used to identify students with reading difficulties with acceptable reliability. Although the reliability of the results were at a very good level overall, the identification was even better in the reading fluency than in reading accuracy and in terms of raw scores than in grade-normative scores. These findings are the first to demonstrate the promise of GBA in assessing reading skills reliably and in a cost-efficient manner in classrooms. Furthermore, the developed GBA is directly applicable to an educational game for successfully supporting reading development of learners with varying levels of reading skill.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Identification of Reading Difficulties by a Digital Game-Based Assessment Technology

Hautala

Heikkilä

Nieminen

et al. 2020

Journal of Educational Computing Research

View full text Add to dashboard Cite

show abstract

“…When dealing with complex problem-solving tasks, the analysis of the processing steps and sequences makes it possible to distinguish successful processing strategies from less successful ones. Ramalingdam and Adams (2018) showed that the reliability and validity of the scale in the field of digital reading can be increased by using log-file data. In the context of their study, only multiple choice items were considered.…”

Section: Increased Reliability: Use Of Log-file Datamentioning

confidence: 99%

Technology and Assessment

Sibberns¹

2020

IEA Research for Education

View full text Add to dashboard Cite

The International Association for the Evaluation of Educational Achievement (IEA) is moving toward computer-based assessment (CBA) in all its internationally conducted large-scale assessment studies. This is motivated by multiple factors, such as the inclusion of more comprehensive measures of the overall construct, increased use of online tools for instruction and assessment in IEA member countries, the need to develop assessments that are more engaging for students, and the hope that CBA will lead to increased efficiency and yield cost savings for participating countries. The transition to CBA has an impact on the design and procedures in both the international study centers and the national study centers. The very high standards in paper-based testing must be preserved throughout the transition steps. This relates not only to the layout, translation, and translation verification steps but also concerns the test administration and all associated procedures. The consistency, accuracy, and completeness of all collected data have to be achieve the same standards as paper-based testing, and the maintenance of the trend measures is of critical importance. The challenges arising from the transition to e-assessment are presented and discussed, including procedures developed to correct for any potential deviations that can be attributed to changes in the delivery mode. CBA provides potential new avenues for data analysis. Log-file and process data are of particular interest to researchers because such data provide an opportunity to increase the validity of the scales and minimize influences irrelevant to the construct being measured.

show abstract

“…When an assessment is delivered via computer, every interaction of the test-taker with the test may be easily recorded, making it possible to collect far more information about student activities during the test than has previously been possible (Baker & Mayer, 1999;Chung & Baker, 2003;Schacter et al, 1999). While it is now possible to record such information, what must be addressed if one is to consider making use of such data is whether it is possible to use them to derive valid measures of the construct (Ramalingam & Adams, 2017). That is, to what extent do each of the final product, and measures of the process, reflect the construct of collaboration?…”

Section: Scoring: What Should Be Scored?mentioning

confidence: 99%

Comparative analysis of student performance in collaborative problem solving: What does it tell us?

Scoular

Eleftheriadou

Ramalingam

et al. 2020

Australian Journal of Education

Self Cite

View full text Add to dashboard Cite

Collaboration is a complex skill, comprised of multiple subskills, that is of growing interest to policy makers, educators and researchers. Several definitions and frameworks have been described in the literature to support assessment of collaboration; however, the inherent structure of the construct still needs better definition. In 2015, the Organisation for Economic Cooperation and Development, in their Programme for International Student Assessment assessed 15-year-old students’ collaborative problem solving achievement, with the use of computer-simulated agents, aiming to address the lack of internationally comparable data in this field. This paper explores what the data from this assessment tell us about the skill, and how these data compare with data from two other assessments of collaboration. Analyses enable comment on the extent to which the three assessments are measuring the same construct, and the extent to which the construct can be covered using computer-based assessments. These investigations generate better understanding of this complex and innovative domain.

show abstract

How Can the Use of Data from Computer-Delivered Assessments Improve the Measurement of Twenty-First Century Skills?

Cited by 10 publications

References 18 publications

Identification of Reading Difficulties by a Digital Game-Based Assessment Technology

Identification of Reading Difficulties by a Digital Game-Based Assessment Technology

Technology and Assessment

Comparative analysis of student performance in collaborative problem solving: What does it tell us?

Contact Info

Product

Resources

About