Evaluating information technology in teaching and learning

Abstract

The effectiveness of instructional technology depends on appropriate evaluation of the technology, with special focus on outcomes. Because evaluation assesses how effective the technology is in enabling learners to master a particular subject, what students learn becomes an important criterion for evaluation. But how and when to assess learning and comprehension is an important and continuing problem. This paper deals with strategic evaluation, which emphasizes technical accuracy, pedagogical soundness, substantive fidelity, integrative flexibility, and cyclic improvement. Effective strategic evaluation is a continuing process-ranging from querying immediate comprehension, to modifying instruction, to assessing long-term effects.The effectiveness of instruction depends on an appropriate evaluation of that instruction. Evaluation has several components, perhaps the most important of which is the evaluation of outcomes. As a result, what students learn becomes an important criterion for evaluation. But how and when to assess learning and comprehension is an important and continuing problem. Do we evaluate just at the time at which the student learns an important concept-say, at the end of a class; or at the end of the course-say, in a final exam; or by considering a portfolio of products that the student develops over a period of years? Clearly, the effective evaluation of instruction is a continuing process, which ranges from queries about immediate comprehension to the assessment of long-term effects.In this paper, I will sometimes appear to refer to instructional software and information technology interchangeably. This is not because I consider the two to be equivalent, but because the evaluation of instructional software has a long history, and it is sometimes simpler to refer to it. However, the use of instructional software is only a small part of the rapidly growing genre of information technologies used in instruction. While instructional software is one type of information technology that can be used in teaching and learning, information technology embraces a wide range of media, including software, CD-ROM, interactive video, and sound. As I survey the new information technologies that are available to educators, I am convinced that effective evaluation is essential, and some of the lessons that we have learned about the effective use of instructional software also apply to all aspects of information technology in education. Moreover, as new and more sophisticated information technologies become available, it is even more critical that Requests for reprints should be addressed to the author at the Department of Psychology, Indiana University, Bloomington, IN 47405 (e-mail: castellan@indiana.edu).appropriate evaluations be done if we are to make intelligent and effective use of the media.It is especially heartening to hear the discussion of evaluation presented in this session. Duncan (1993), Ransdell (1993), and Welsh (1993) have described not only the importance of evaluation, but the critical need for ...

Animal Use in Undergraduate Psychology Programs

“…This result suggests that it was not additional time spent reviewing the study guide, taking quizzes, or mere additional time dealing with the content of the course that affected performance. Rather, the time spent completing the specific exercises translated 180 into gains in learning that material; this is congruent with Castellan's (1993) study.…”

Technologically Assisted Instruction and Student Mastery, Motivation, and Matriculation

“…The intervention also illustrates the importance of systematically pursuing feedback about the use of technology in teaching (Castellan, 1993;Duncan. 1993).…”

A method for evaluating the effectiveness of educational software

“…Seven classes, with a total of 327 students, participated as experimental groups, and three classes (from Tufts), with a total of 63 ing of impediments to learning probability and statistics. Like Castellan (1993), we recognize the need for cyclic improvement.…”

“…Many of the smaller studies lack a robust enough design to permit transfer of results. Welsh (1993), Duncan (1993), Ransdell (1993), and Castellan (1993) have offered a set of guidelines for im-. proving evaluation of educational software.…”