Evaluation criteria and the effectiveness of instructional technology in higher education

Abstract

No significant differences" is the most frequently quoted conclusion of any survey comparing the effectiveness [1] of alternative instructional technologies [2,3]. While this effectiveness conclusion does not differ for the research reported upon in this paper the other purposes of this paper are to discuss alternative criteria for measuring effectiveness and to analyse potential causes of the finding of no significant differences.The paper is divided into three major sections. In the first section an analysis of the impact of comparability of technologies, organizational factors, student characteristics, and material quality on technology effectiveness is presented. The second section is devoted to a description of cognitive (measurements of knowledge gains) and noncognitive (attitudes and time to complete material) criteria for evaluation. The third section contains a summarization of research findings for a variety of technologies including: television, radio, programmed instruction, computer-assisted instruction and effects of alterations in class size, changes in course length, and utilization of graduate students as instructors. Problems in the Evaluation of Instructional Technologies COMPARABILITY OF TECHNOLOGIESTo determine the relative cognitive effectiveness of instructional technology comparisons are usually made with conventional instruction. Unfortunately there is no clear definition of conventional instruction. It is also difficult to specify precise experimental controls for the studies. For example, in a comparison of televised and conventional classroom instruction, should we use the same instructor with the same materials in both media? Or should we allow total flexibility in course construction to utilize the differences in the media? What size of class should be allowed for the conventional instruction alternative? Briggs et al. (1967) pointed to other difficulties in comparison studies including: the use of intact classes and the +' Prepared for the Economics Education Project, Heriot-Watt University, Edinburgh, Scotland. The author would like to thank the referees for Higher Education for their helpful comments on an earlier draft of this paper.