Many Hats: The Methods and Roles of the Program Evaluator

Abstract

Program evaluation often casts the social scientist in multiple roles that require the use of various methods and skills to deliver the product or products desired by the hiring agency. Social scientists engaged in evaluation research often find their roles being expanded beyond those articulated in their contracts. Balancing these numerous demands while maintaining professional integrity can be both challenging and rewarding in terms of professional growth and expansion of one's professional network. This article focuses on the experiences of one research team working with a law enforcement agency in a midsized city in the Southwest during the first two years of the implementation of a Weed and Seed program. Nimbleness of method, expansion of the evaluator's role, and the reality of unexpected challenges are the primary lessons to be drawn from this experience.

Minding the Gap: Using an Applied Research Center to Improve Student Learning

“…The skill development and experiences expressed by Joe and Kelly earlier are crucial for getting a head start in the job market. Liberal arts graduates often use research-related skills on the job market as noted by Ballard and Daniel (2016), Hirsch and Quartaroli (2009), Bills (2008), and Spalter-Roth (2007). ARC student employees expressed that they developed these skills directly in their work in the Center, not in the classroom.…”

“…The relationship between the cost versus the benefits of higher education are often a direct concern for parents, students, and professionals whose work involves higher education, especially for those with liberal arts degrees (Adamuti-Trache et al 2006; Humphreys 2014; McPherson 1998; Paino 2014; Rowen 2016; Skinner and Lawson 2006). These concerns have led to increased focus for describing the value of liberal arts degrees, in terms of social (Ballard and Daniel 2016; Hollway 2005; Paino 2014; Sandahl 2015; Spalter-Roth et al 2010) and economic benefits (Adamuti-Trache et al 2006; Humphreys 2014; Rowen 2016; Tubbs 2013) in addition to finding alternative ways to measure and produce desired student learning outcomes (Bills 2008; Gullion and Ellis 2014; Hirsch and Quartaroli 2009; Holbrook and Chen 2017; Korsching and Peter 2007; Lehnerer 2008; Stolley et al 2017). Previous research has also focused on documenting student gains during two specific time periods: during degree attainment (Bills 2008; Gullion and Ellis 2014; Lehnerer 2008; McKinney and Naseri 2011) and into the workforce (Stolley et al 2017; Ballard and Daniel 2016; Bandini et al 2016; Spalter-Roth 2007; Spalter-Roth and Van Vooren 2010).…”

“…Course-based and service-learning strategies, in the form of research projects, are often drawn on by faculty and instructors to provide students with hands-on active learning experiences. These experiences have shown to be especially beneficial in solidifying classroom learning (Bandini et al 2016; Gullion and Ellis 2014) because these activities allowed students to apply their training to authentic community concerns (Hirsch and Quartaroli 2009; Holbrook and Chen 2017; Lehnerer 2008). As social science and humanities graduates become critical thinkers, their discipline-specific expertise and professional skills become a resource for a variety of institutions, such as in nonprofit organizations.…”

Building Research Capacity in a Mobile Psychiatric Rehabilitation Program: Service Learning in a University-Agency Collaboration

“…The instructor’s role is more than that of an educator; it also includes actively managing and supervising the program evaluation design process. Parallel to the multiple roles of a program evaluator discussed by Hirsch and Quartaroli (2009), the process of designing an evaluation that incorporates the time, staffing, budgetary, and technological constraints of the agency context can be frustrating for students and challenge the problem-solving skills of the instructor. However, the complicated nature of this applied research project may be what prepares students to implement these skills in their future practice settings, and as such, represents one model of building evaluation capacity for current social service agencies and future practitioners.…”

Program Evaluation Standards for Utility Facilitate Stakeholder Internalization of Evaluative Thinking in the West Virginia Clinical Translational Science Institute

“…This yields the opportunity for increased evaluation use (Alkin & King, 2017). In turn, the evaluative process benefits from an evaluator playing both the role of a trusted friend who is not afraid to tell the truth (Rallis & Rossman, 2000) and the role of a teacher who passes judgment (Schwandt, 2001), though they most certainly serve in many additional roles (e.g., Hirsch & Quartaroli, 2009;Mathison, 1991;Morabito, 2002).…”