Using Social Policy Research for Reducing Social Problems: An Empirical Analysis of Structure and Functions

Abstract

Investigating three samples of social policy research in The Netherlands, the authors analyzed the conditions and functions influencing utilization (impact). Interorganizational context, intergroup relations, and role interaction were found to relate to utilization. The impact of social policy research upon organizational decisions is cognitive, communicative, and diagnostic. The cognitive function correlates negatively with publishing for scholarly audiences. Of the communicative function, six feedback strategies were evaluated with targeting scoring highest on impact. In the diagnostic function, creating consensus among decision-making groups has the greatest impact on decision making. The results suggest the latent operation of a professional paradigm of social policy research, distinct from the disciplinary social science paradigm.

Current Empirical Research on Evaluation Utilization

“…Simulation studies (n = 21), were almost exclusively experimental, generally employing random allocation of subjects to groups. Some of these studies used blocking variables to allocate subjects to groups (e.g., Newman, Brown, Rivers, & Glock 1983); some used repeated measures designs (e.g., Study samples varied considerably in size (from 1 to 4,500; Mdn = 60) and in the unit of analysis (generally the individual respondent, but sometimes the evaluation project per se -Brickell, 1976;Carter, 1971;Leviton & Boruch, 1983;Rossman, Hober, & Ciarlo, 1979;Van de Vail & Bolas, 1982; or sometimes the domain of the evaluation- Alkin, Kosecoff, Fitz-Gibbon, & Seligman, 1974;Bigelow& Ciarlo, 1976;David, 1978;Kennedy, 1983Kennedy, , 1984Tash&Stahler, 1982;Williams & Bank, 1984). Education respondents were typically teachers, principals, board members, and/or school system or district office administrators.…”

“…For example, program planning, summative judgements of program impact, staff scheduling, and room arrangements (e.g., Glasman, 1984;Kennedy, 1984;Lorenzen & Braskamp, 1978;Maher, 1982;Sproull & Zubrow, 1981;Yeh, 1980). Some studies operationalized use in terms of extent of use, impact, and helpfulness of the evaluation (e.g., Alkin & Stecher, 1983;Dickey, 1980;Goldberg, 1978;Jordan, 1977;Ory & Braskamp, 1980;Patton et al, 1977;Rich, 1979;Van de Vail & Bolas, 1982;Williams & Banks, 1984). Other studies had respondents make discrete decisions based on given evaluative information (Granville, 1977;Heldt, Braskamp, & Filbeck, 1973).…”

“…Evaluation quality was typically defined as methodological sophistication, type of approach to the evaluation problem, or the intensity of the evaluation activities. Weeks (1979), Van de Vail and Bolas (1982), and Siegal and Tuckel (1985) reported that increased methodological sophistication served to inhibit use (as defined in terms of use index variables and observations). Similarly, Yeh (1980) reported that instructional decisions were more likely to be based on less sophisticated teacher-made tests.…”

“…Most studies found that evaluations that reflected knowledge of the context in which the evaluations were to be used, appealed to preferences of the decisionmak ers, sought consensus about the evaluation problem, or demonstrated insight into program operations and decisionmaking, were associated with higher levels of use (Da‰on & D'Amico, 1985;Osterlind 1979;Rich, 1979;Rossman et al, 1979;Van de Vail & Bolas, 1982). Evaluations that ignored these issues were associated with relatively low levels of use (David, 1978;FΊorio et al, 1979;McGowan, 1976;Neigher, 1979;Sproull & Zubrow, 1981;Wholey & White, 1973).…”

The art of co-production of knowledge in environmental sciences and management: lessons from international practice

“…Engagement and communication activities are widely recognized as the cornerstones of producing usable information (Kothari et al 2005;Oh and Rich 1996;van de Vall and Bolas 1982) and co-production of knowledge (Dilling and Lemos 2011;Lemos and Morehouse 2005;Reed et al 2014). Not only does deep engagement with end-users tend to increase the likelihood that the information will be used (Evely et al 2010), there is evidence that it improves the quality of decisions resulting from the process (Shirk et al 2012;Beierle 2002).…”