A Systematic Literature Review of Methodology of Learning Evaluation Based on Item Response Theory in the Context of Programming Teaching

Santos, Jucelio Soares dos; Andrade, Wilkerson L.; Brunet, João; Melo, Monilly Ramos Araujo

doi:10.1109/fie44824.2020.9274068

Cited by 6 publications

(4 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To quantify students' KSAs, researchers often use static outcomebased measures and summative assessments, such as scores and grades [25,29,30]. However, these data capture only the final submitted form of the assignment, which a student may have iteratively refined to make it optimal using the results of the automated assessment.…”

Section: Outcome-based Vs Behavioral Measuresmentioning

confidence: 99%

Comparison of Three Programming Error Measures for Explaining Variability in CS1 Grades

Švábenský,

Pankiewicz,

Zhang

et al. 2024

Proceedings of the 2024 on Innovation and Technology in Computer Science Education v. 1

View full text Add to dashboard Cite

Section: Outcome-based Vs Behavioral Measuresmentioning

confidence: 99%

Comparison of Three Programming Error Measures for Explaining Variability in CS1 Grades

Švábenský,

Pankiewicz,

Zhang

et al. 2024

Proceedings of the 2024 on Innovation and Technology in Computer Science Education v. 1

View full text Add to dashboard Cite

“…IRT is a unique approach to identifying the relationship between students' latent ability and the potential to choose certain answers (Kim et al, 2021). The rapid increase of study makes educational research conduct IRT to validate test instruments in classical tests and some programming in study fields even at the university level (Santos et al, 2020).…”

Section: Utbk As the University Entrance Examinationmentioning

confidence: 99%

Washback Effects of English Reading Test in UTBK: Students’ Insights

Sabillawati

Ardiana

2022

ELER

View full text Add to dashboard Cite

Ujian Tulis Berbasis Komputer (UTBK) is a high-stakes test for students. They strive to pass the test and be accepted in their targeted institution. As one of the subjects involved in UTBK, the English test surely has influenced English teaching & learning in 12th-grade classrooms. This impact is known as the washback effect. While most studies investigate the washback effect on a test as a whole subject, little is known about the effect on the particular part of the test. Thus, this case study explores the washback effect of the English reading test in UTBK on the students’ perceptions, learning motivation, learning strategies, and learning content. The data were collected through online questionnaires distributed to six participants from different senior high schools. The deductive analysis found that the test has positive and negative effects seen from the students’ insights. UTBK can avoid cheating and integrate students’ English competence, but they are uncertain that it can improve their communicative competence. This study might enrich educational studies in English language teaching in general and language assessment in particular.

show abstract

“…Yet, it can also be used to obtain systematic information about the 'difficulty' of concepts and the distribution of the respective competencies among students. This can be done based on the code created by the students as an outcome of the learning process, regarding certain attributes of the code as manifestations of latent psychometric constructs according to the principles of IRT [47][48] [62]. The occurrence of certain concepts like loops or conditional statements can be considered as satisfiability on certain items (e.g., "the existence of loops").…”

Section: Introductionmentioning

confidence: 99%

An Item Response Theory Analysis of Algorithms and Programming Concepts in App Inventor Projects

Alves

Wangenheim

Hauck

et al. 2021

Anais Do I Simpósio Brasileiro De Educação Em Computação (EDUCOMP 2021)

View full text Add to dashboard Cite

Computing education is often introduced in K-12 focusing on algorithms and programming concepts using block-based programming environments, such as App Inventor. Yet, learning programming is a complex process and novices struggle with several difficulties. Thus, to be effective, instructional units need to be designed regarding not only the content but also its sequencing taking into consideration difficulties related to the concepts and the idiosyncrasies of programming environments. Such systematic sequencing can be based on large-scale project analyses by regarding the volition, incentive, and opportunity of students to apply the relevant program constructs as latent psychometric constructs using Item Response Theory to obtain quantitative ?difficulty? estimates for each concept. Therefore, this article presents the results of a large-scale data-driven analysis of the demonstrated use in practice of algorithms and programming concepts in App Inventor. Based on a dataset of more than 88,000 App Inventor projects assessed automatically with the ANON rubric, we perform an analysis using Item Response Theory. The results demonstrate that the easiness of some concepts can be explained by their inherent characteristics, but also due to the characteristics of App Inventor as a programming environment. These results can help teachers, instructional and curriculum designers in the sequencing, scaffolding and assessment design of programming education in K-12.

show abstract

A Systematic Literature Review of Methodology of Learning Evaluation Based on Item Response Theory in the Context of Programming Teaching

Cited by 6 publications

References 39 publications

Comparison of Three Programming Error Measures for Explaining Variability in CS1 Grades

Comparison of Three Programming Error Measures for Explaining Variability in CS1 Grades

Washback Effects of English Reading Test in UTBK: Students’ Insights

An Item Response Theory Analysis of Algorithms and Programming Concepts in App Inventor Projects

Contact Info

Product

Resources

About