Transformation-based diagnosis of student programs for programming tutoring systems

Abstract

A robust technology that automates the diagnosis of students' programs is essential for programming tutoring systems. Such technology should be able to determine whether programs coded by a student are correct. If a student's program is incorrect, the system should be able to pinpoint errors in the program as well as explain and correct the errors. Due to the difficulty of this problem, no existing system performs this task entirely satisfactorily, and this problem still hampers the development of programming tutoring systems. This paper describes a transformation-based approach to automate the diagnosis of students' programs for programming tutoring systems. Improved control-flow analysis and data-flow analysis are used in program analysis. Automatic diagnosis of student programs is achieved by comparing the student program with a specimen program at the semantic level after both are standardized. The approach was implemented and tested on 525 real student programs for nine different programming tasks. Test results show that the method satisfies the requirements stated above. Compared to other existing approaches to automatic diagnosis of student programs, the approach developed here is more rigorous and safer in identifying student programming errors. It is also simpler to make use of in practice. Only specimen programs are needed for the diagnosis of student programs. The techniques of program standardization and program comparison developed here may also be useful for research in the fields of program understanding and software maintenance.

Using Targeted Feedback to Address Common Student Misconceptions in Introductory Programming: A Data-Driven Approach

“…The first type of feedback system uses artificial intelligence (AI) techniques to analyze students’ programs and generate personalized feedback for students (Barnes & Stamper, 2010; Rivers & Koedinger, 2017; Xu & Chee, 2003). With such an intelligent feedback component, an automated assessment system becomes an intelligent tutoring system that can not only grade students’ programs but also provide automated feedback.…”

Análise de aprendizagem a partir de códigos-fontes e uma proposta de seleção automática de métricas de avaliação / Analysis of learning from source-codes and a proposal for automatic selection of evaluation metrics

“… Medição de esforço e complexidade de codificação [Curtis et al 1979b]  Predição de desempenhos [Xu andChee 2003, Naude et al 2010]  Análise de estilo de programação [Rees 1982, Berry andMeekings 1985]  Avaliação por métricas de software [Hung et al 1993  Reconhecimento de indícios de plágios [Neves et al 2017b]  Reconhecimento de rubricas [Oliveira et al 2018]  Recomendação de Atividades [Oliveira et al 2013]  Análise de proficiência em programação [Kumar et al 2014]  Análise de dificuldades de aprendizagem [Neves et al 2017b] Entre as soluções tecnológicas de análise de aprendizagem de programação baseadas em análise estática já propostas, destacamos: as métricas de Halstead e McCabe [Halstead 1977, Curtis et al 1979b, Berry and Meekings 1985, a avaliação de habilidades de programação por métricas de software de [Hung et al 1993], o sistema de recomendação de atividades conforme dificuldades de aprendizagem de [Oliveira et al 2013], a análise de dificuldades por métricas de software de [Oliveira et al 2017b], a avaliação de como estudantes de programa-ção aprendem a partir da análise de seus códigos de programação [Blikstein et al 2014] e a análise de proficiência em programação realizada pelo sistema SCALE de [Kumar et al 2014].…”