Automated Grading of Class Diagrams

Bian, Weiyi; Alam, Omar; Kienzle, Jörg

doi:10.1109/models-c.2019.00106

Cited by 27 publications

(18 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The purpose of the evaluation is to determine whether any of these classifiers are useful for practitioners (e.g., Would a course staff member choose to use this tool?). Bian et al [3] evaluated their model in comparison to an instructor grading the same submission. We do likewise, considering the human grades to be ground truth and evaluate each classifier against it.…”

Section: Discussionmentioning

confidence: 99%

“…Krusche et al developed an interactive online learning system for modeling where the grading is done by humans [12]. Bian et al [3] have implemented an automatic class diagram grader within TouchCore, a software design and modeling tool with touchscreen support. Like Umple, it can be used to build class diagrams.…”

Section: Related Workmentioning

confidence: 99%

“…In this section, we explain how we obtained grades for the student submissions using TouchCore's model grading option [3].…”

Section: Touchcore Grading Pipelinementioning

confidence: 99%

“…In this paper, we propose an approach based on a simple heuristic and machine learning (ML) that can help categorize simple domain model submissions according to their quality. The proposed approach predicts whether (i) a submission is above a quality threshold (i.e., a binary classifier); or (ii) its approximate letter grade out of A, B, C, D, and F. We contrast this lightweight approach with the complex rule-based one used in TouchCore [3], a modeling application. Better understanding the situation of students' models as soon as they are submitted can help course staff estimate grading effort and tailor their lectures and tutorials to better suit student need.…”

mentioning

confidence: 99%

“…Approach: Our approach uses the output of a simple heuristic as training data for five ML classifiers for both the binary classifier and letter grade prediction. We evaluate the performance of the different classifiers on student submissions against the ground truth results from human graders, and compare with TouchCore's complex rulebased method [3], which grades class diagrams by comparing them against an ideal solution. For the comparison, we first use Touch-Core grading output by itself and then combined with ML, and finally ML with a combination of heuristic and TouchCore output for training data.…”

mentioning

confidence: 99%

See 4 more Smart Citations

Automatic assessment of students' software models using a simple heuristic and machine learning

Boubekeur

Mussbacher

McIntosh

2020

Proceedings of the 23rd ACM/IEEE International Conference on Model Driven Engineering Languages and Systems: Companion Proceedi

View full text Add to dashboard Cite

Software models are increasingly popular. To educate the next generation of software engineers, it is important that they learn how to model software systems well, so that they can design them effectively in industry. It is also important that instructors have the tools that can help them assess students' models more effectively. In this paper, we investigate how a tool that combines a simple heuristic with machine learning techniques can be used to help assess student submissions in model-driven engineering courses. We apply our proposed technique to first identify submissions of high quality and second to predict approximate letter grades. The results are comparable to human grading and a complex rule-based technique for the former and surprisingly accurate for the latter. CCS CONCEPTS • Software and its engineering → Unified Modeling Language (UML); • Applied computing → Education; • Computing methodologies → Machine learning.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

“…In this section, we explain how we obtained grades for the student submissions using TouchCore's model grading option [3].…”

Section: Touchcore Grading Pipelinementioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Automatic assessment of students' software models using a simple heuristic and machine learning

Boubekeur

Mussbacher

McIntosh

2020

Proceedings of the 23rd ACM/IEEE International Conference on Model Driven Engineering Languages and Systems: Companion Proceedi

View full text Add to dashboard Cite

show abstract

Automated generation and correction of diagram‐based exercises for Moodle

Gómez‐Abajo,

Guerra,

de Lara

2023

Comp Applic In Engineering

View full text Add to dashboard Cite

One of the most time‐consuming task for teachers is creating and correcting exercises to evaluate students. This is normally performed by hand, which incurs high time costs and is error‐prone. A way to alleviate this problem is to provide an assistant tool that automates such tasks. In the case of exercises based on diagrams, they can be represented as models to enable their automated model‐based generation for any target environment, like web or mobile applications, or learning platforms like Moodle. In this paper, we propose an automated process for synthesizing five types of diagram‐based exercises for the Moodle platform. Being model‐based, our solution is domain‐agnostic (i.e., it can be applied to arbitrary domains like automata, electronics, or software design). We report on its use within a university course on automata theory, as well as evaluations of generality, effectiveness and efficiency, illustrating the benefits of our approach.

show abstract

Towards standarized benchmarks of LLMs in software modeling tasks: a conceptual framework

Cámara,

Burgueño,

Troya

2024

Softw Syst Model

View full text Add to dashboard Cite

The integration of Large Language Models (LLMs) in software modeling tasks presents both opportunities and challenges. This Expert Voice addresses a significant gap in the evaluation of these models, advocating for the need for standardized benchmarking frameworks. Recognizing the potential variability in prompt strategies, LLM outputs, and solution space, we propose a conceptual framework to assess their quality in software model generation. This framework aims to pave the way for standardization of the benchmarking process, ensuring consistent and objective evaluation of LLMs in software modeling. Our conceptual framework is illustrated using UML class diagrams as a running example.

show abstract

Automated Grading of Class Diagrams

Cited by 27 publications

References 11 publications

Automatic assessment of students' software models using a simple heuristic and machine learning

Automatic assessment of students' software models using a simple heuristic and machine learning

Automated generation and correction of diagram‐based exercises for Moodle

Towards standarized benchmarks of LLMs in software modeling tasks: a conceptual framework

Contact Info

Product

Resources

About