Predicting Performance in an Introductory Programming Course by Logging and Analyzing Student Programming Behavior

Abstract

Use policyThe full-text may be used and/or reproduced, and given to third parties in any format or medium, without prior permission or charge, for personal research or study, educational, or not-for-prot purposes provided that:• a full bibliographic reference is made to the original source • a link is made to the metadata record in DRO • the full-text is not changed in any way The full-text must not be sold in any format or medium without the formal permission of the copyright holders.Please consult the full DRO policy for further details. Abstract-The high failure rates of many programming courses means there is a need to identify struggling students as early as possible. Prior research has focused upon using a set of tests to assess the use of a student's demographic, psychological and cognitive traits as predictors of performance. But these traits are static in nature, and therefore fail to encapsulate changes in a student's learning progress over the duration of a course. In this paper we present a new approach for predicting a student's performance in a programming course, based upon analyzing directly logged data, describing various aspects of their ordinary programming behavior. An evaluation using data logged from a sample of 45 programming students at our University, showed that our approach was an excellent early predictor of performance, explaining 42.49% of the variance in coursework marks -double the explanatory power when compared to the closest related technique in the literature.

No tests required

“…However studies to date are limited by a lack of verification, and a tendency to yield inconsistent results [23]. Additionally the previously researched predictors require the use of lengthy tests to gather predictive data.…”

A New Metric to Quantify Repeated Compiler Errors for Novice Programmers

“…Watson et al [20] also found a significant correlation between the EQ and the scores of students in their data (r = 0.44). They proposed an extended algorithm called Watwin which takes into account the time a student spends in overcoming a particular error.…”

Predição de Zona de Aprendizagem de Alunos de Introdução à Programação em Ambientes de Correção Automática de Código

“…O autor verificou que o EQ possui uma correlação com a nota do aluno. Nesse mesmo caminho, [Watson et al 2013] criou uma derivação da métrica EQ, levando em consideração o tempo que o aluno levava para consertar o erro. O algoritmo proposto por Watson, chamado Watwin Score, obteve uma acurácia média de 68,8% ao longo do curso e atingiu 75,6% no final do curso, enquanto no trabalho de Jadud os resultados foram de 55,8% de acurácia média e de 60,0% ao final do curso, ambos para predição de desempenho [Watson et al 2013].…”