Automatic Generation of Programming Exercises

Abstract

Massive training of developers following the growing demands of the information technology industry requires teachers to automate their repetitive tasks. For training courses on programming, it is promising to use automatic generation and automatic grading of exercises that require a student to write a program. This article discusses the general scheme for constructing a programming exercises generator and identifies two classes of exercises, the generation of which can be automated: converting notation into code and converting data formats. Several examples of programming exercise generators are discussed. The experience of using exercise generators for the Python programming course is briefly described.

Classification of Program Texts Represented as Markov Chains with Biology-Inspired Algorithms-Enhanced Extreme Learning Machines

“…The Digital Teaching Assistant system automates the massive Python programming course at MIREA-Russian Technological University. DTA automatically generates [6] unique programming exercises of 11 different types and automatically checks the programs submitted by students [9,10]. Brief descriptions of programming exercise types generated by DTA are listed in Table 1.…”

“…Code to vector transformation is also required during classifier training and prediction-making [35], so in this section we briefly describe vectorization of program texts submitted by students. Several approaches to source code to vector transformation exist [35,36], but the DTA analytics module uses its own vectorization algorithm, which respects the specifics of the generators of unique programming exercises [6,10].…”

“…Besides, methods based on preliminary code vectorization exist [5] that employ unsupervised machine learning techniques while analyzing programs written by students. A more reliable approach to cheating prevention is using unique sets of programming tasks generated for every student [6,7]. Automatic programming exercise generation with support for automatic grading has tremendous potential for teaching programming to a large and diverse audience [8].…”

“…The Digital Teaching Assistant system (DTA) [9,10], which automates the Python programming course at MIREA-Russian Technological University, offers means for Python programming tasks generation of various types, as well as means for automatic assessment of student submissions. The DTA system consists of three modules, namely: the core module responsible for unique programming tasks generation for every student using the generate and test approach [6], and which is also responsible for submitted solutions checking; the web application module, allowing students to interact with the core module via a web-based user interface (UI) [9]; the analytics module, allowing programming instructors to track progress statistics of every student and to analyze the large number of submitted programs with the help of machine learning.…”

“…Machine learning algorithms allow solving complex problems by analyzing large datasets with minimal human intervention; such algorithms automatically discover patterns hidden in datasets. When the DTA system generates unique programming exercises of various types for every student [6,10], students can barely cheat. However, every task of the given type can be solved using different methods, and automatic discovery of the most commonly used methods using a clustering-based approach helped teachers to identify knowledge gaps of students by the end of semester.…”