Assessing collaborative problem-solving skills among elementary school students

Rojas, Matías; Nussbaum, Miguél; Chiuminatto, Pablo; Guerrero, Orlando; Greiff, Samuel; Krieger, Florian; Westhuizen, Lindie van der

doi:10.1016/j.compedu.2021.104313

Cited by 17 publications

(5 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…• and various uses of clickstream data from computer log files (i.e., keystrokes, mouse movements) to investigate and draw inferences about response times and interactions (Wise and Kong, 2005[36]; Wise, 2017 [37]; Reis Costa et al, 2021 [38]; Salles, Dos Santos and Keskpaik, 2020 [9]; Michaelides, Ivanova and Nicolaou, 2020 [39]; Goldhammer et al, 2021 [40]; Deribo, Goldhammer and Kroehne, 2022 [41]).…”

Section: Figure 1: Collecting Data On Response Processesmentioning

confidence: 99%

“…Measures of respondent performance are increasingly used to supplement, enhance or replace conventional test scores (Mislevy et al, 2014[70]; von Davier et al, 2017 [71]; Rojas et al, 2021[72]; Stoeffler et al, 2020[73]). This includes log data on item response times (Li, Banerjee and Zumbo, 2017 [74]; Reis Costa et al, 2021 [38]; Ercikan, Guo and He, 2020 [3]; Deribo, Goldhammer and Kroehne, 2022 [75]) and clickstream data about response processes within the item -including type and number of 'actions' and 'events' that can be used to infer the strategies that respondents use in tacking items (Salles, Dos Santos and Keskpaik, 2020 [9]; Goldhammer et al, 2021[8]; Ercikan, Guo and He, 2020 [3]). While item response times provide a signal about variation in response processes and their relationship with test scores, response data collected on performance within items, i.e., from clickstream, eye tracking and think aloud provides a richer source of information to inform and extend measures of item performance.…”

Section: Measures Of Performancementioning

confidence: 99%

“…In the deliberate uses of process data by design to measure student performance, upstream design processes require considerable investment in iterative, evidenced design and validation to ensure user experience and responses support process model assumptions (Kane and Mislevy, 2017 [5]). That design process is quite different to retrospective data mining of process, for example, that use time stamp data or information on within item log events to yield large-scale supplementary data on student performance from more conventional, product-oriented assessment designs (e.g., (Ercikan, Guo and He, 2020 [3]; Reis Costa et al, 2021 [38])). However, whether it is for process data by design, or in retrospective data mining, any use of process data to make performance related inferences require validation arguments, models and evidence (Goldhammer et al, 2021[8]).…”

Section: Measures Of Performancementioning

confidence: 99%

See 2 more Smart Citations

The uses of process data in large-scale educational assessments

2023

OECD Education Working Papers

View full text Add to dashboard Cite

The digital transition in educational testing has introduced many new opportunities for technology to enhance large-scale assessments. These include the potential to collect and use log data on test-taker response processes routinely, and on a large scale. Process data has long been recognised as a valuable source of validation evidence in assessments. However, it is now being used for multiple purposes across the assessment cycle. Process data is being deliberately captured and used in large-scale, standardized assessmentsmoving from viewing it as a 'by-product' of digital assessment, to its use 'by design' to extend understanding of test-taker performance and engagement. While these techniques offer significant benefits, they also require appropriate validation practices to ensure that their use supports reliable inferences and does not introduce unintended negative consequences. EDU/WKP(2023)1  5 THE USES OF PROCESS DATA IN LARGE-SCALE EDUCATIONAL ASSESSMENTS Unclassified Table of contentsAcknowledgements .

show abstract

Section: Figure 1: Collecting Data On Response Processesmentioning

confidence: 99%

Section: Measures Of Performancementioning

confidence: 99%

Section: Measures Of Performancementioning

confidence: 99%

See 1 more Smart Citation

The uses of process data in large-scale educational assessments

2023

OECD Education Working Papers

View full text Add to dashboard Cite

show abstract

“…It has provided a substantial body of theory and evidence related to computer-based assessments of the skills involved in the human-to-agent approach (i.e., H-A approach), which makes test-takers collaborate with a couple of computer agents to tackle simulative problems. A great deal of subsequent theoretical and empirical studies on ColPS have followed, drawing on the established framework of the PISA 2015 ColPS assessment and the data that were generated (e.g., Chang et al, 2017;Child & Shaw, 2019;Graesser et al, 2018;Herborn et al, 2017;Kang et al, 2019;Rojas et al, 2021;Swiecki et al, 2020;Tang et al, 2021;Wu et al, 2022).…”

mentioning

confidence: 99%

Behavioral patterns in collaborative problem solving: a latent profile analysis based on response times and actions in PISA 2015

Han,

Krieger,

Borgonovi

et al. 2023

Large-scale Assess Educ

Self Cite

View full text Add to dashboard Cite

Process data are becoming more and more popular in education research. In the field of computer-based assessments of collaborative problem solving (ColPS), process data have been used to identify students’ test-taking strategies while working on the assessment, and such data can be used to complement data collected on accuracy and overall performance. Such information can be used to understand, for example, whether students are able to use a range of styles and strategies to solve different problems, given evidence that such cognitive flexibility may be important in labor markets and societies. In addition, process information might help researchers better identify the determinants of poor performance and interventions that can help students succeed. However, this line of research, particularly research that uses these data to profile students, is still in its infancy and has mostly been centered on small- to medium-scale collaboration settings between people (i.e., the human-to-human approach). There are only a few studies involving large-scale assessments of ColPS between a respondent and computer agents (i.e., the human-to-agent approach), where problem spaces are more standardized and fewer biases and confounds exist. In this study, we investigated students’ ColPS behavioral patterns using latent profile analyses (LPA) based on two types of process data (i.e., response times and the number of actions) collected from the Program for International Student Assessment (PISA) 2015 ColPS assessment, a large-scale international assessment of the human-to-agent approach. Analyses were conducted on test-takers who: (a) were administered the assessment in English and (b) were assigned the Xandar unit at the beginning of the test. The total sample size was N = 2,520. Analyses revealed two profiles (i.e., Profile 1 [95%] vs. Profile 2 [5%]) showing different behavioral characteristics across the four parts of the assessment unit. Significant differences were also found in overall performance between the profiles.

show abstract

“…Namun, telah terbukti bahwa penggunaan video game secara moderat dapat meningkatkan perkembangan kognitif siswa (Rojas et al, 2021), kinerja akademik (Ibáñez et al, 2020), dan kondisi fisik (Williams & Ayres, 2020). Selain itu, video game juga memfasilitasi inisiasi latihan fisik dan olahraga untuk siswa dengan performa olahraga yang lebih rendah dan menghasilkan kepatuhan, berkat motivasi yang dihasilkan (Rogers et al, 2020).…”

unclassified

Pengembangan media video tutorial permainan tradisional gobak sodor bagi guru PJOK sekolah dasar

Sesanti,

Sumarno,

Imawati

2023

pej

View full text Add to dashboard Cite

Abstrak: Tujuan dari penelitian ini adalah mengembangkan dan menguji media pembelajaran video tutorial permainan tradisonal gobak sodor bagi guru PJOK sekolah dasar. Penelitian ini menggunakan model pengembangan ADDIE dengan metode 5 tahapan. Pada tahap validasi produk dilakukan 2 validator, ahli media dan materi. Selain itu responden penelitian guru dan peserta didik. Instrumen yang digunakan, Wawancara, Observasi, Lembar Validasi Produk, dan Lembar Kuesioner responden. Hasil penelitian video tutorial permainan tradisional gobak sodor menunjukkan hasil penilaian memenuhi kriteria “Valid dan “Sangat Valid” serta kategori penilaian “Layak” dan “Sangat Layak” sehingga dapat dijadikan bahan ajar dalam pembelajaran Pendidikan Jasmani di SDN Sumberasri 01 Kecamatan Nglegok Kabupaten Blitar. Abstract: The purpose of this research is to develop and test video tutorial learning media for traditional gobak sodor games for elementary school PJOK teachers. This research uses the ADDIE development model with a 5-stage method. At the product validation stage, 2 validators were carried out, media and material experts. In addition, research respondents were teachers and students. The instruments used, Interview, Observation, Product Validation Sheet, and Respondent Questionnaire Sheet. The results of the research on video tutorials for traditional gobak sodor games show that the assessment results meet the criteria of "Valid and "Very Valid" as well as the assessment categories "Feasible" and "Very Feasible" so that they can be used as teaching materials in learning Physical Education at SDN Sumberasri 01 Nglegok District, Blitar Regency.

show abstract

Assessing collaborative problem-solving skills among elementary school students

Cited by 17 publications

References 47 publications

The uses of process data in large-scale educational assessments

The uses of process data in large-scale educational assessments

Behavioral patterns in collaborative problem solving: a latent profile analysis based on response times and actions in PISA 2015

Pengembangan media video tutorial permainan tradisional gobak sodor bagi guru PJOK sekolah dasar

Contact Info

Product

Resources

About