Modelling Assessment Rubrics through Bayesian Networks: a Pragmatic Approach

Rubric-based Learner Modelling via Noisy Gates Bayesian Networks for Computational Thinking Skills Assessment

“…Therefore, this work extends our previous model [15] by adding the possibility of modelling a set of supplementary skills necessary, in conjunction with the skills under examination, to succeed in the assigned tasks. To this goal, it is essential to extend the nosy-OR approach in [13] to combine disjunctive and conjunctive relations between skills since.…”

“…1) Noisy-OR Gates: This section briefly introduces noisy-OR gates and their use in learner modelling. We refer to [13] and [15] for a more detailed discussion. A typical representation of the noisy-OR network structure, introducing n auxiliary variables (also called inhibitor nodes), is shown in Fig.…”

“…Similar advantages also concern the inference. In [15], we adopted a solution to set up a general approach for translating assessment rubrics into interpretable BN-based learner models with a complexity compatible with real-time assessment.…”

“…Therefore, although we do not explicitly model skill interactions, they can be captured in the hierarchy of complex behaviours identified by the rubric. However, in our previous work [15], this hierarchy of competencies was only indirectly modelled. This produced assessments assigning larger probabilities to higher-level competence, in contrast with the assumption that when a competence of a certain level is possessed, all lower-level competencies are also owned.…”

“…In the model presented here, this counter-intuitive behaviour has been eliminated by imposing the constraints codified by the rubric through observed auxiliary nodes that do not modify the network structure and retain the previous model's relative simplicity. Moreover, considering that assessment rubrics are usually limited to the competence components under assessment, hereafter also called target skills, the modelling approach in [15] may have led to oversimplified learner models, unable to grasp the actual causes of a failure which is not always due to the absence of the target skills. It may, for instance, follow from deficiencies in other skills necessary for the specific task, hereafter referred to as supplementary skills.…”