A Taxonomy and Mapping of Computer-Based Critiquing Tools

Abstract

Critics have emerged in recent times as a specific tool feature to support users in computermediated tasks. These computer-supported critics provide proactive guidelines or suggestions for improvement to designs, code, and other digital artifacts. The concept of a critic has been adopted in various domains, including medical, programming, software engineering, design sketching, and others. Critics have been shown to be an effective mechanism for providing feedback to users. We propose a new critic taxonomy based on extensive review of the critic literature. The groups and elements of our critic taxonomy are presented and explained collectively with examples, including the mapping of 13 existing critic tools, predominantly for software engineering and programming education tasks to the taxonomy. We believe this critic taxonomy will assist others in identifying, categorizing, developing, and deploying computer-supported critics in a range of domains.

Classifying and Evaluating Assessment Feedback Practices

“…In terms of the development of taxonomical structures in the assessment and feedback areas, recent examples include [37], [38], [39], [40] and [41]. However, these examples attempt to categorise very particular elements only, such as computer-based critiquing tools, adaptive feedback, group supervision, formative assessment for mathematical modelling problems and the characteristics of student peer mentors.…”

WebTA: Automated iterative critique of student programming assignments

“…Students use the tool iteratively to improve their code before submission. Ali, Hosking, and Grundy [2] categorize the Java Critiquer as an analytic critic, using text-based pattern matching through instructor authored rules to provide explanations and suggestions. The Java Critiquer influenced the static assessment portion of our application that evaluates the students' programming style.…”

Critiquing Antipatterns In Novice Code

“…Critiquers are similar to autograders, but they focus on providing highly interactive and targeted feedback to student programmers [4]. While these systems often perform testing, they make strong use of the instructor's domain knowledge to identify patterns and formulate meaningful responses.…”

“…Hosking, and Grundy [3,5] categorize the Java Critiquer as an analytic critic, using text-based pattern matching through instructor authored rules to provide explanations and suggestions. The Java Critiquer influenced the static assessment portion of our application that evaluates students' programming style.…”