Cluster Analysis to Estimate the Difficulty of Programming Problems

Intisar, Chowdhury Md; Watanobe, Yutaka

doi:10.1145/3274856.3274862

Cited by 18 publications

(6 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Saito and Watanobe (2020) proposed a learning path recommendation system based on a learner's ability charts by means of an RNN. Intisar and Watanobe (2018a) proposed a method for the classification of OJ programmers based on rule extraction from a self-organising feature map, cluster analysis to estimate the difficulty of programming problems (Intisar and Watanobe, 2018b), and classification of programming problems based on topic modelling (Intisar et al, 2019). Teshima and Watanobe (2018) presented bug detection methods for the feedback system of an OJ system.…”

Section: Related Researchmentioning

confidence: 99%

Code completion for programming education based on deep learning

Terada

Watanobe

2021

IJCISTUDIES

Self Cite

View full text Add to dashboard Cite

In solving programming problems, it is difficult for beginners to create a program from scratch. One way to navigate this difficulty is to suggest the next word following an incomplete program. In the present study, we propose a method for code completion characterised by two principal elements: the prediction of the next within-vocabulary word and the prediction of the next referenceable identifier. For the prediction of within-vocabulary words, a neural language model based on an LSTM network with an attention mechanism is proposed. Additionally, for the prediction of referenceable identifiers, a model based on a pointer network to a given incomplete program is proposed. For evaluation of the proposed method, source code accumulated in an online judge system is used. The results of the experiment demonstrate that in both statically and dynamically typed languages, the proposed method can predict the next word to a high degree of accuracy.

show abstract

Section: Related Researchmentioning

confidence: 99%

Code completion for programming education based on deep learning

Terada

Watanobe

2021

IJCISTUDIES

Self Cite

View full text Add to dashboard Cite

show abstract

“…Jing et al [18] introduced a vocabulary learning model that calculates the incorrect classification cost for the prediction of source code defects. Various ML approaches [19][20][21] have been proposed for classification, recommendation, and estimation problems. Alreshedy et al [22] presented an ML-based language model for classifying source code snippets based on the programming language.…”

Section: Background and Literature Reviewmentioning

confidence: 99%

Source Code Assessment and Classification Based on Estimated Error Probability Using Attentive LSTM Language Model and Its Application in Programming Education

2020

Self Cite

View full text Add to dashboard Cite

The rate of software development has increased dramatically. Conventional compilers cannot assess and detect all source code errors. Software may thus contain errors, negatively affecting end-users. It is also difficult to assess and detect source code logic errors using traditional compilers, resulting in software that contains errors. A method that utilizes artificial intelligence for assessing and detecting errors and classifying source code as correct (error-free) or incorrect is thus required. Here, we propose a sequential language model that uses an attention-mechanism-based long short-term memory (LSTM) neural network to assess and classify source code based on the estimated error probability. The attentive mechanism enhances the accuracy of the proposed language model for error assessment and classification. We trained the proposed model using correct source code and then evaluated its performance. The experimental results show that the proposed model has logic and syntax error detection accuracies of 92.2% and 94.8%, respectively, outperforming state-of-the-art models. We also applied the proposed model to the classification of source code with logic and syntax errors. The average precision, recall, and F-measure values for such classification are much better than those of benchmark models. To strengthen the proposed model, we combined the attention mechanism with LSTM to enhance the results of error assessment and detection as well as source code classification. Finally, our proposed model can be effective in programming education and software engineering by improving code writing, debugging, error-correction, and reasoning.

show abstract

“…Aprender programação requer muita prática, em grande parte através de questões de codificação [1,10,15,21,23,29,30,34,44]. Para um instrutor, a correção manual das soluções elaboradas pelos estudantes, além de desgastante, é por vezes demorada [2,9,29,30].…”

Section: Introductionunclassified

“…Estas não dispõem de informações que possam indicar o quão fácil será sua resolução por um estudante [10,29]. Logo, para que o sorteio aleatório de questões seja equânime, faz-se necessário um mecanismo capaz de classificar a facilidade de novas questões antes de serem apresentadas aos estudantes, utilizando para isso somente os dados disponíveis no momento de cadastro [15,44], ou seja, somente as informações disponibilizadas pelo instrutor.…”

Section: Introductionunclassified

Uso de atributos de código para classificação da facilidade de questões de codificação

Lima¹,

Carvalho²,

Oliveira³

et al. 2021

Anais Do I Simpósio Brasileiro De Educação Em Computação (EDUCOMP 2021)

View full text Add to dashboard Cite

No ensino de programação, é comum o uso de Ambientes de Correção Automática de Código (ACACs). Esses apresentam uma alta diversidade de exercícios de programação que requerem que o estudante elabore um código como solução. Contudo, um obstáculo é classificar a facilidade ou dificuldade desses exercícios de modo que sejam apresentados conforme o nível de conhecimento do aluno, ou para que durante uma avaliação formulada por sorteio aleatório de questões, os alunos possam receber exercícios com nível de dificuldade semelhantes. Nesse sentido, este trabalho apresenta um método para classificar automaticamente o grau de facilidade de questões de codificação em Python, usando para isso atributos extraídos de códigos de solução. Foram analisadas 354 questões, agrupadas em 07 conjuntos segundo os tópicos abordados, que foram previamente definidos na ementa de uma disciplina de introdução à programação. Essas questões foram aplicadas em exames presenciais de turmas ministradas entre os anos de 2017 e 2019. Entre os resultados, verificou-se que a facilidade das questões pode ser estimada por meio de atributos do código de solução dos instrutores, com f1-score de 0,94 e acurácia de 0,92. Além disso, o conjunto de atributos relevantes para classificação variou conforme o tópico de programação abordado pelas questões.

show abstract

Cluster Analysis to Estimate the Difficulty of Programming Problems

Cited by 18 publications

References 12 publications

Code completion for programming education based on deep learning

Code completion for programming education based on deep learning

Source Code Assessment and Classification Based on Estimated Error Probability Using Attentive LSTM Language Model and Its Application in Programming Education

Uso de atributos de código para classificação da facilidade de questões de codificação

Contact Info

Product

Resources

About