The Utility of Evaluation Research for Administrative Decision-Making

Towards systematic reviews that inform health care management and policy-making

“…In our systematic review, 17 studies reported in 20 articles met the inclusion criteria. The 17 studies included seven focused on health care managers (three employed case studies, [13][14][15][16] two employed surveys, 17,18 and two employed interviews 19,20 ) and 10 focused on health care policy-makers (four employed case studies, [21][22][23][24] and six employed interviews [25][26][27][28][29][30][31][32] ). Data were extracted from the studies by two individuals working independently and then independently checked by a third reviewer (the lead author).…”

“…Our systematic review of studies of decision-making by health care managers and policy-makers has led us to a number of observations about the state of research evidence for each of the two groups. Compared to studies of health care policy-makers, [21][22][23][24][25][26][27][28][29][30][31][32] the studies of managers are: [13][14][15][16][17][18][19][20] fewer in number (seven studies compared to 10); less likely to use more than one method of data collection to examine research use (one study -a case study that drew on interviews supplemented by document analysis -compared to three); roughly equally likely to not describe adequately the sampling and measurement methods that were used (two studies of health care managers met all three of the methodological criteria and five did not meet them, whereas one study of health care policymakers met all three of the methodological criteria, three partially met them, and six did not meet them); and less likely to yield findings consistent across a number of contexts (only two factors emerged in more than one study of managers and no factors emerged in more than two studies, whereas six factors emerged in two studies of policy-makers, two factors emerged in three studies, one factor emerged in four studies, and one factor emerged in six studies).…”

Evaluation as a Means of Organizational Learning

“…McNeece, DiNitto, and Johnson (1983), in their study of 42 health directors of cotnmunity based organizations, reported that while 83% of the study participants discussed having been involved in evaluation research, only 25% noted that data on program effectiveness influenced their operational decisions. Weiss (1980), based on the reports of over 150 decision makers in community mental health agencies, reported that 33% of respondents did not use evaluation data in their decision making; another 10% of the sample, while answering that they utilized evaluation data, could not describe how it was used.…”