Evaluating Information System Effectiveness - Part I: Comparing Evaluation Approaches

Abstract

While the use and costs of Management Information Systems (MIS) have become highly visible, tittle attention has been paid to assessing and communicating system effectiveness. Evaluation of system effectiveness is difficult due to its multidimensionality. its quantitative and quafitative aspects, and the multiple. and often conflicting, evaluator viewpoints. This article provides an overview of what system effectiveness means and how it should be measured. It is the first of two articles to appear in consecutive issues of the MIS. Quarterly. Starting with a definition of system effectiveness, this article examines evaluation of system effectiveness in terms of a conceptual hierarchy of system objectives. The hierarchy is used to discuss problems in, and recommendations for, evaluating system effectiveness, and to compare MIS evaluation approaches. The second article characterizes and compares the evaluator viewpoints on system effectiveness for decision makers in several functional groups involved in MIS implementation-user, MIS, internal audit, and management. The second article recommends several MIS evaluation approaches for incorporating multiple dimensions and multiple evaluator viewpoints into evaluations of information system effectiveness.

Information Technology and Corporate Strategy: A Research Perspective

“…In particular, we need to study the impact of different forms of information technology, as categorized in Figure 1, on the factors that constrain the performance of individual and group decision makers. A good deal of effort has already been expended studying the link between information systems and managerial performance [15,16,25]. More headway could be made in this important area if a common characterization of information technology was used across studies and if the intervening variables that act as boundaries to rationality, and hence managerial performance, were identified and included in the study.…”

The influence of service quality in information system success model as an antecedent of mobile learning in education organisations: case study in Jordan

“…Initially, user satisfaction was examined in this context where the IS success model comprised of six dimensions, which are system quality, information quality, system use, user satisfaction, individual impact and organisational impact (DeLone and McLean, 1992). In the model, system quality encapsulates different system characteristics namely system reliability, online response time (Swanson, 1974), system accuracy, database content (Emery, 1971), completeness, flexibility of the system and ease of use (Hamilton and Chervany, 1981). Meanwhile information quality consists of quality of the information system output -pioneering authors conducted an evaluation of information value on the basis of its characteristics including its accuracy, timeliness, relevance, formatting (Ahituv, 1980), informativeness, usefulness (Gallagher, 1974), sufficiency, understandability, reliability, comparability, devoid of bias, and quantitativeness (King and Epstein, 1983).…”

Student User Satisfaction with Web-based Information Systems in Korean Universities

“…Information system evaluation on how well they are effectively used, therefore, has become one of very important research topics (Harrison and Rainer, 1996). Many of previous studies suggested that user satisfaction can be a viable variable measuring the system effectiveness (Baroudi et al, 1988;Conrath and Mignen, 1990;Ginzberg, 1978;Hamilton and Chervany, 1981;Ives and Olson, 1984;Powers and Dickson, 1974). In other words, one can conclude that if the users are well satisfied with the system as they use it, then the system will be working fine.…”