Questionnaires as a software evaluation tool

Abstract

This paper reports on a study investigating the strengths and weaknesses of questionnaires as software evaluation tools. Two major influences on the usefulness of questionnaire-based evaluation responses are examined: the administration of the questionnaire, and the background and experience of the respondent. Two questionnaires were administered to a large number of students in an introductory programming class. The questionnaires were also given to a group of more experienced users (including course proctors). Respondents were asked to evaluate the text editor used in the class along a number of dimensions; evaluation responses were solicited using a number of differentThe order of authors is not significant.Several people put in a lot of work on this project, most prominently Amy Geoffroy, Jan Graham, and Claire O'Malley. We are also indebted to Professor Kenneth Bowles who offered us a full and generous cooperation in our attempts to invade his Pascal course with our research. A number of other people also contributed to the design and wording of the questionnaires, including:

MyWallMate: An Investigation into the use of Mobile Technology in Enhancing Student Engagement

“…This method is suitable as it is a quick and easy approach that is able to collect relevant data from a cohort that spans multiple modules and it is an effective method for evaluating user interfaces and software (Caltenco et al 2012;Fitzpatrick et al 2011;Root and Draper 1983).…”

A usability comparison of three alternative message formats for an SMS banking service

“…The use of questionnaires to evaluate user interfaces has a long history (Root and Draper, 1983;LaLomia and Sidowski, 1990). The questionnaire used a seven-point Likert format that ranged from ''strongly agree'' (1) to ''strongly disagree'' (7).…”

An Empirical Evaluation of Usability for SICCAT Tool

“…Totally 35 papers were identified, 15 papers were rejected as not relevant to this research and included 20 papers finally. According to [2], Questionnaires have long been used to evaluate user interfaces.…”