Assessment and Plagiarism

Lancaster, Thomas; Robins, Anthony; Fincher, Sally

doi:10.1017/9781108654555.015

Cited by 5 publications

(4 citation statements)

References 1,544 publications

(67 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The SOLO taxonomy typically classifies responses to a programming problem into four sequential levels [71]: (1) prestructural responses lack understanding of the problem or show knowledge unrelated to the problem; (2) unistructural responses provide a description for a small part of the code; (3) multistructural responses provide a line by line description of most of the code; (4) relational responses summarize the code in terms of its purpose. A general agreement is that solving an unfamiliar problem requires relational responses, but there is disagreement on responses for more specific problems [34]. One primary reason may be that the level of responses to a problem relies on students' prior knowledge and abilities [45].…”

Section: Literature Reviewmentioning

confidence: 99%

“…(3) multistructural responses provide a line by line description of most of the code; (4) relational responses summarize the code in terms of its purpose. A general agreement is that solving an unfamiliar problem requires relational responses, but there is disagreement on responses for more specific problems [34]. One primary reason may be that the level of responses to a problem relies on students' prior knowledge and abilities [45].…”

Section: Introductionmentioning

confidence: 99%

“…Researchers have used Bloom's taxonomy [10] and the Structure of Observed Learning Outcomes (SOLO) taxonomy [8] to understand programming proficiencies [34,61,71]. The current study focuses on the SOLO taxonomy because it has been widely used for evaluating the responses to problems requiring different programming skills [16,38,68].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Utilizing programming traces to explore and model the dimensions of novices' code‐writing skill

Zhang

Paquette

Pinto

et al. 2023

Comp Applic In Engineering

View full text Add to dashboard Cite

Studies have found that most novice programmers have low proficiency in writing code. However, it is unclear what subskills compose code writing and which subskills novice programmers struggle with. This study utilizes programming traces to identify latent subskills that constitute code writing so that teachers can offer specific instruction on the weak subskills. Data were collected from an undergraduate course teaching introductory computer science in Java. Six hundred and fourteen students made submissions to homework programming questions in a web‐based learning system. Based on the submission traces, we computed 11 features related to correctness and time students spent on their submissions. We conducted an exploratory factor analysis on two‐thirds of students selected randomly and identified four factors. The first factor, code style proficiency, was mainly related to code style errors. The second, syntactic proficiency, concerned compiler errors. The third is semantic proficiency, which concerns runtime and logic errors. The fourth, syntactic debugging proficiency, concerned the success rate and time required for fixing compiler and code style errors. A confirmatory factor analysis conducted on the remaining one‐third of the data supported the four‐factor structure. The factor model showed measurement invariance between the data set where the model was developed and two new datasets, one from the same sample but collected at a different time point and another from a different sample and context (onsite course vs. online course). The factors were related to prior programming abilities, programming language familiarity, and future exam performance. These associations provided validity evidence for the factor model.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Utilizing programming traces to explore and model the dimensions of novices' code‐writing skill

Zhang

Paquette

Pinto

et al. 2023

Comp Applic In Engineering

View full text Add to dashboard Cite

show abstract

“…Educational assessment is a crucial aspect of training (Lancaster, Robins, & Fincher, 2019). It enables teachers (instructors) to better understand the actions of their students (trainees).…”

Section: Why Is Student Assessment Necessary?mentioning

confidence: 99%

Student assessment in cybersecurity training automated by pattern mining and clustering

et al. 2022

View full text Add to dashboard Cite

Hands-on cybersecurity training allows students and professionals to practice various tools and improve their technical skills. The training occurs in an interactive learning environment that enables completing sophisticated tasks in full-fledged operating systems, networks, and applications. During the training, the learning environment allows collecting data about trainees’ interactions with the environment, such as their usage of command-line tools. These data contain patterns indicative of trainees’ learning processes, and revealing them allows to assess the trainees and provide feedback to help them learn. However, automated analysis of these data is challenging. The training tasks feature complex problem-solving, and many different solution approaches are possible. Moreover, the trainees generate vast amounts of interaction data. This paper explores a dataset from 18 cybersecurity training sessions using data mining and machine learning techniques. We employed pattern mining and clustering to analyze 8834 commands collected from 113 trainees, revealing their typical behavior, mistakes, solution strategies, and difficult training stages. Pattern mining proved suitable in capturing timing information and tool usage frequency. Clustering underlined that many trainees often face the same issues, which can be addressed by targeted scaffolding. Our results show that data mining methods are suitable for analyzing cybersecurity training data. Educational researchers and practitioners can apply these methods in their contexts to assess trainees, support them, and improve the training design. Artifacts associated with this research are publicly available.

show abstract

Academic Integrity for Computer Science Instructors

Lancaster

2023

Higher Education Computer Science

View full text Add to dashboard Cite

Assessment and Plagiarism

Cited by 5 publications

References 1,544 publications

Utilizing programming traces to explore and model the dimensions of novices' code‐writing skill

Utilizing programming traces to explore and model the dimensions of novices' code‐writing skill

Student assessment in cybersecurity training automated by pattern mining and clustering

Academic Integrity for Computer Science Instructors

Contact Info

Product

Resources

About