Psychometric Evaluation of the Cybersecurity Concept Inventory

PoulsenSeth,; HermanGeoffrey, L; PetersonPeter, A H; GolaszewskiEnis,; GortiAkshita,; OlivaLinda,; ScheponikTravis,; ShermanAlan, T

doi:10.1145/3451346

Cited by 5 publications

(2 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To answer RQ1, we want to understand what level of student knowledge Proof Blocks questions assess, and how accurately it assesses that knowledge. 2PL has been used widely in psychometrics and has been used within computer science education mostly for validation of concept inventories [10,18,19,37]. 2PL is a good fit for our needs because it provides a way to model the probability of each student answering each question correctly as a function of a question's difficulty and discrimination.…”

Section: Psychometricsmentioning

confidence: 99%

Evaluating proof blocks problems as exam questions

et al. 2022

Self Cite

View full text Add to dashboard Cite

P roof Blocks is a novel software tool which enables students to write mathematical proofs by dragging and dropping prewritten lines into the correct order, rather than writing a proof completely from scratch. We used Proof Blocks problems as exam questions for a discrete mathematics course with hundreds of students, allowing us to collect thousands of student responses to Proof Blocks problems. Using this data, we provide statistical evidence that Proof Blocks are easier than written proofs, which are typically very difficult. We also show that Proof Blocks problems provide about as much information about student knowledge as written proofs. Survey results show that students believe that the Proof Blocks user interface is easy to use, and that the questions accurately represent their ability to write proofs.

show abstract

Section: Psychometricsmentioning

confidence: 99%

Evaluating proof blocks problems as exam questions

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…CIs are an increasingly popular tool for CS education researchers. The last decade has seen a Cambrian explosion of CIs for CS topics such as CS1 [Kaczmarczyk et al 2010], CS2 [Wittie et al 2017], algorithms [Farghally et al 2017], recursion [Hamouda et al 2017], data structures [Porter et al 2019], digital logic , operating systems [Webb and Taylor 2014], and cybersecurity [Poulsen et al 2021]. The development of these CIs has proved useful in demonstrating both the existence and frequency of misconceptions.…”

Section: Introductionmentioning

confidence: 99%

A Grounded Conceptual Model for Ownership Types in Rust

Crichton,

Gray,

Krishnamurthi

2023

Proc. ACM Program. Lang.

View full text Add to dashboard Cite

Programmers learning Rust struggle to understand ownership types, Rust’s core mechanism for ensuring memory safety without garbage collection. This paper describes our attempt to systematically design a pedagogy for ownership types. First, we studied Rust developers’ misconceptions of ownership to create the Ownership Inventory, a new instrument for measuring a person’s knowledge of ownership. We found that Rust learners could not connect Rust’s static and dynamic semantics, such as determining why an ill-typed program would (or would not) exhibit undefined behavior. Second, we created a conceptual model of Rust’s semantics that explains borrow checking in terms of flow-sensitive permissions on paths into memory. Third, we implemented a Rust compiler plugin that visualizes programs under the model. Fourth, we integrated the permissions model and visualizations into a broader pedagogy of ownership by writing a new ownership chapter for The Rust Programming Language , a popular Rust textbook. Fifth, we evaluated an initial deployment of our pedagogy against the original version, using reader responses to the Ownership Inventory as a point of comparison. Thus far, the new pedagogy has improved learner scores on the Ownership Inventory by an average of 9

show abstract