A system to grade computer programming skills using machine learning

Srikant, Shashank; Aggarwal, Varun

doi:10.1145/2623330.2623377

Cited by 68 publications

(48 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Outras pesquisas, dentre os quais citamos [Silva et al 2014, Srikant and Aggarwal 2014, Choudhury et al 2016, Yin et al 2015, buscam agrupar ou classificar soluções de código. Em cada uma destas pesquisasé adotadado um conjunto diferente de técnicas.…”

Section: Diversasunclassified

“…Nos trabalhos de Choudhury e Yin [Choudhury et al 2016, Yin et al 2015 são utilizados o algoritmo de agrupamento OPTICS, a métrica ABC [Fitzpatrick 2000] e similaridade baseada em distância deárvores, com o intuito de auxiliar, através de dicas, estudantes matriculados em cursos de Engenharia de Software a produzir código com maior 'qualidade'. No trabalho de Srikant, em [Srikant and Aggarwal 2014],é proposto um meio de aplicar aprendizagem de máquina (regressão linear, random forests e Support Vector Machines -SVMs) para aprender quais propriedades que são levadas em consideração na avaliação de um exercício de programação.…”

Section: Diversasunclassified

See 1 more Smart Citation

Uma abordagem adaptativa para gerar agrupamento de códigos em disciplinas de programação introdutória

Barbosa¹,

Costa²,

Brito³

2017

Anais Do XXVIII Simpósio Brasileiro De Informática Na Educação (SBIE 2017)

View full text Add to dashboard Cite

Abstract. Despite the importance of introductory programming disciplines, it is quite common to find problems. In such environments, we easily find unmotivated students with some doubts and that do not understand basic concepts. Monitoring each one of the students is not trivial because there are many students and it is necessary to observe many characteristics of each code submited for practical activities. In this research, clustering of codes to minimize the effort of evaluation is investigated. The results vary from reasonable to perfect concordances, considering the semiautomatic evaluations obtained with the clustering and the expert evaluations.Resumo. Apesar da importância das disciplinas introdutórias de programação, e bastante comum encontrar problemas de ensino e aprendizagem. Facilmente, em tais ambientes, encontramos alunos desmotivados, com dúvidas não esclarecidas e que não entendem conceitos básicos. O acompanhamento individualizado dos alunos nãoé trivial, pois existem muitos alunos eé necessário observar muitas características de cada código proposto nas diversas atividades práticas adotadas. Nesta pesquisa, investigou-se o agrupamento de soluções, visando minimizar o esforço de avaliação despendido. As concordâncias obtidas nos resultados variam de razoável a perfeita, considerando as avaliações semiautomáticas obtidas com os agrupamentos e as avaliações de especialistas. IntroduçãoOs períodos iniciais dos cursos daárea de computação, em geral, englobam diferentes disciplinas que possuem como foco o estudo de algoritmos e a implementação de programas. Facilmente encontramos nestas disciplinas alunos desmotivados, com dúvidas não esclarecidas e que não conseguem entender os conceitos necessários para a prática de programação. Além disso, muitos dos alunos aprovados não possuem as competências necessárias ao curso eà vida profissional [McCracken et al. 2001].

show abstract

Section: Diversasunclassified

Uma abordagem adaptativa para gerar agrupamento de códigos em disciplinas de programação introdutória

Barbosa¹,

Costa²,

Brito³

2017

Anais Do XXVIII Simpósio Brasileiro De Informática Na Educação (SBIE 2017)

View full text Add to dashboard Cite

show abstract

“…For example, some systems were developed to automatically grade programming source texts. Some of these systems take comparable approaches to ours as similarities between students' and key responses are in the center of interest (e.g., [13], [14]). Some also apply clustering methods on these similarities (e.g., [15]).…”

Section: Related Workmentioning

confidence: 99%

Using and Improving Coding Guides for and by Automatic Coding of PISA Short Text Responses

Zehner

Goldhammer

Sälzer

2015

2015 IEEE International Conference on Data Mining Workshop (ICDMW)

View full text Add to dashboard Cite

Abstract-We propose and empirically evaluate a theoretical framework of how to use coding guides for automatic coding (scoring) and how, in turn, automatic coding can enhance the use of coding guides. We adopted a recently described baseline approach to automatically classify responses. Well-established coding guides from PISA, comprising reference responses, and its German sample from 2012 were used for evaluation. Ten items with 41,990 responses at total were analyzed. Results showed that (1) responses close to the cluster centroid constitute prototypes, (2) automatic coding can improve coding guides, while (3) the proposed procedure leads to unreliable accuracy for small numbers of clusters but promising agreement to human coding for higher numbers. Further analyses are still to be done to find the optimal balance of the implied coding effort and model accuracy.

show abstract

“…Substantial progress has been made on automated grading and feedback systems in several restricted domains, including essay evaluation using natural language processing (NLP) [1,33], computer program evaluation [12,15,29,32,34], and mathematical proof verification [8,19,21].…”

Section: Introductionmentioning

confidence: 99%

Mathematical Language Processing

Lan

Vats

Waters

et al. 2015

Proceedings of the Second (2015) ACM Conference on Learning @ Scale

View full text Add to dashboard Cite

While computer and communication technologies have provided effective means to scale up many aspects of education, the submission and grading of assessments such as homework assignments and tests remains a weak link. In this paper, we study the problem of automatically grading the kinds of open response mathematical questions that figure prominently in STEM (science, technology, engineering, and mathematics) courses. Our data-driven framework for mathematical language processing (MLP) leverages solution data from a large number of learners to evaluate the correctness of their solutions, assign partial-credit scores, and provide feedback to each learner on the likely locations of any errors. MLP takes inspiration from the success of natural language processing for text data and comprises three main steps. First, we convert each solution to an open response mathematical question into a series of numerical features. Second, we cluster the features from several solutions to uncover the structures of correct, partially correct, and incorrect solutions. We develop two different clustering approaches, one that leverages generic clustering algorithms and one based on Bayesian nonparametrics. Third, we automatically grade the remaining (potentially large number of) solutions based on their assigned cluster and one instructor-provided grade per cluster. As a bonus, we can track the cluster assignment of each step of a multistep solution and determine when it departs from a cluster of correct solutions, which enables us to indicate the likely locations of errors to learners. We test and validate MLP on real-world MOOC data to demonstrate how it can substantially reduce the human effort required in large-scale educational platforms.

show abstract

A system to grade computer programming skills using machine learning

Cited by 68 publications

References 11 publications

Uma abordagem adaptativa para gerar agrupamento de códigos em disciplinas de programação introdutória

Uma abordagem adaptativa para gerar agrupamento de códigos em disciplinas de programação introdutória

Using and Improving Coding Guides for and by Automatic Coding of PISA Short Text Responses

Mathematical Language Processing

Contact Info

Product

Resources

About