This article reviews the technique of verbal protocol analysis and gives a profile of its use within software engineering research over the last two decades. An overview is given of the procedures used in verbal protocol analysis, and commonly-found difficulties in the application of the technique by researchers are described. The article reports on published efforts to develop tools to automate the procedures. A review of the literature shows trends in the use of the verbal protocol analysis in software engineering research from the 1980s to the present. Recurring themes of its purpose within software engineering research are identified, including the comparison of the behaviours of subjects with differing levels of expertise and the identification of effective software comprehension strategies. Advances and problems with the development of a general-purpose encoding scheme for verbal protocol analysis appropriate to a range of domains within software engineering are described.

No abstract

A number of agile practices are included in software engineering curricula, including test-driven development. Continuous integration often is not included, despite it becoming increasingly common in industry to code, test, and integrate at the same time. This paper describes a study whereby software engineering undergraduates were given a short intensive experience of test-driven development with continuous integration using an environment that imitated a typical industrial circumstance. Assessment was made of students' agile experience rather than of project deliverables, using a novel set of process measures that examined students' participation and performance in agile testing. Results showed good participation by student pairs, and clear understanding of agile processes and configuration management. Future work will investigate automation of the assessment of continuous integration and configuration management server data.

No abstract

This study describes the context for the development of a tool to formatively assess information technology skills of students. The tool provides a reliable and valid assessment of word processing competency, utilizing automation to apply the test instrument via the Microsoft Office package. Tests can be designed directly by tutors, and delivered via a network. Evaluation of the tests suggests that immediate automated testing is preferred by students compared to a traditional written test. There is evidence that the tool improves the IT skills of its users, whereas a traditional written test has no such beneficial effect. Introduction and backgroundIn the United Kingdom, the Report of the National Committee of Inquiry into Higher Education in 1997 recognised the need for Higher Education institutions to provide a programme for all students which included information technology (IT) as one of a set of key skills. The Quality Assurance Agency was established as one mechanism for guiding institutions towards acceptable specifications of knowledge, understanding and skills for different programmes. The importance of information technology is demonstrated by its inclusion within the "transferable skills" component of subject benchmark statements published by the QAA. The set of information technology skills required tends to evolve in response to computing hardware and software developments, the changing set of skills brought by new entrants from school experience, and the expectations of tutors and professional bodies (Martin, 1997). Additionally, distinct disciplines have subject-specific requirements of IT. Nevertheless, the core components of most programmes include skills in word processing, email, information retrieval from databases and the Internet, and competence in the basic use of a personal computer. These skills are taught either by means of discrete IT classes or by embedding the teaching of IT skills at relevant points within subject-specific classes. The efficacy of the different approaches has been a popular topic for researchers (for example, see Martin, 1997). Less well researched is the most appropriate and effective method of assessing such skills.