A Framework for the Evaluation of CASE Tool Learnability in Educational Environments

Abstract

The aim of the research is to derive a framework for the evaluation of Computer Aided Software Engineering (CASE) tool learnability in educational environments. Drawing from the literature of Human Computer Interaction and educational research, a framework for evaluating CASE tool learnability in educational environments is derived. The two main differences between this framework and existing evaluation frameworks is that 1) it is specific to educational environments and 2) it recognizes the significance of contextual factors such as learning environment and teaching and learning methodologies in evaluating learnability. The framework is oriented towards learnability and is customisable. It helps to define the important issues related to CASE tool learnability, and allows conclusions to be drawn about all influential factors on learning in the given context.A detailed case study describes the application of the framework to evaluate the learnability of Rational Rose, a CASE tool used in an undergraduate Systems Analysis and Design course.

Evaluating the TESTAR tool in an industrial case study

“…Although learning (level 2) and performance (level 3) criteria are conceptually related [9], this threat was not mitigated because environmental variables of the hands-on learning process could not be monitored. Only working diaries were self-reported by the trainees.…”

“…In our case, although the learnability evaluation was based on a four-level strategy [9] that we have used before, some of the threats could not be fully mitigated, at least, for the two first levels (Reaction and Learning). This is because most of the collected data was based on trainee's responses.…”

“…The same questionnaire is applied at point A, after the course but before the hands-on learning, and at point B, after the hands-on learning. The questions have been taken from [9] where the authors are analyzing the learnability of CASE tools. They have been divided into 7 categories to separate di↵erent aspects of the tool.…”

Towards an Experimental Framework for Measuring Usability of Model-Driven Tools

“…If we focus our research on MDD tools, little work has been published about their usability. We can only find usability frameworks defined for evaluating usability in Computer Aided Software Engineering (CASE) tools in general, such as the work of Senapathi [8]. However, frameworks defined for CASE tools are not always valid for MDD tools.…”

An empirical approach for evaluating the usability of model-driven tools

“…For example Senapathi [28] focuses mainly on the learnability of UML CASE tools in an educational environment. His studies involve questionnaires and observation of 150 students in the classroom.…”