The Role of Evaluation in Ensuring Excellence in Communications and Information Technology Initiatives

Abstract

This article presents an approach to the evaluation of communications and information technology (CIT) programmes in which the evaluation makes a positive contributio n to ensuring excellence in the programme. After a brief introductio n outlining changing attitude s to the commissioning of evaluations of CIT policy initiatives in the last 20 years, the article explores the politicise d purposes of evaluations , the kinds of knowledge that sponsors may be seeking, and the need for evaluations to use a mixture of quantitativ e and qualitativ e methods. Characteristi c problematic features of CIT programmes are then outlined, relating both to their products and their processes. This section deals with cultural differences, due to the diverse disciplinary , institutional , and often national, backgrounds of their programme teams. It is suggested that the resulting complexity makes it essential for evaluators to intervene in ways that prevent these problems from undermining the potential of CIT programmes to achieve excellence. In the nal section, there is a short review of some of the relevant literature on programme evaluation, followed by an account of recent evaluation studies of CIT programmes. The article ends with detailed discussion of the components of the suggested model of evaluation to ensure excellence, derived from the author's experience as an evaluator and in uenced by her reading of the research literature .We have reached a point in time when it is possible to envisage that communications and information technology (CIT) could begin to deliver some of the educational goals which have been promised for the last two decades. The technology itself is now much more user-friendly; the community of software engineers and developers has the bene t of experi-

Tools for the job: a report of two surveys of information and communications technology training and use for literacy in primary schools in the West of England

“…NOF training, although largely welcomed by teachers, had areas for improvement, such as providing time for assimilation (OFSTED 2001). For example, the gender divide in confidence and perceived competence noted in 1998 had narrowed by 2001, but training should be evaluated with this in mind to ensure that ICT's potential for teaching and learning in communication and collaborative work is fully exploited (Somekh 2001).…”

Design characteristics that affect speed of information access and clarity of presentation in an electronic neuroanatomy atlas

“…These are all higher orders of abstract thinking. Whether multimedia learning tools actually promote better learning outcomes at higher levels is an ongoing focus of research (Somekh, 2001;Anderson and Kanuka, 2003;Eysenbach, 2003). Until recently, there has been no established peer-review system for electronic learning objects, although there is good agreement on the need for one.…”

“…Furthermore, the efficacy and quality of currently available programs vary widely (Kim et al, 2003). Therefore, as the demand for electronic education grows, so too does the demand for systematic evaluation so as to improve its quality (Somekh, 2001).…”

Investigating the Effectiveness of Computer Simulations in the Teaching of “Atomic Structure and Bonding”

“…This fact is supported by Means and Olson [15] who noted that technology engaged students in challenging and authentic learning. Students from the treatment group have also asserted that they preferred computer simulations-based instruction over regular instruction; this finding lends support to Somekh [20] who has indeed highlighted that computer technology can provide strong visual elements that would be readily welcomed by teachers and students.…”

“…Data collected during the lessons in the treatment group have also indicated that the computer simulations were better appreciated than regular instruction by the students. Besides, all students of the treatment group have stated in the questionnaire that they have liked the visual effects provided by computer simulations; indeed, Somekh [20] has earlier reported that computer technology can provide strong visual elements that would be readily welcomed by teachers and students. In addition, Papert [21] has also stated that early exposure to science through visualization promotes independent learning by using the child's intrinsic curiosity, and by integrating science into his understanding of the world.…”