Facilitating the evaluation of complexity in the public sector: learning from the NHS in Scotland

Abstract

It is necessary for public managers to be able to evaluate programmes in the context of complexity. This article offers key learning and reflections based on the experience of facilitating the evaluation of complexity with a range of public sector partners in Scotland. There have been several articles that consider evaluating complexity and theory-based approaches to evaluation; however, the literature is scarce when it comes to actually facilitating such approaches within the public sector. The article discusses the main steps used when facilitating outcomes planning with stakeholders and identifies some key challenges and learning in relation to undertaking this work. The article will be of interest to those that want to know more about the core components of outcomes-based evaluation and, moreover, to those that will undertake this type of facilitation within the public sector as part of organisational development and professional development.

The sustainability of knowledge brokerage of the mental health improvement outcomes framework in Scotland: a follow-up analysis

“…This impeded the development of meaningful action around monitoring and evaluation. Ironically, implementing an outcomes-based approach that allows for the mapping of partnership contributions to programme outcomes requires pragmatic conversations to be had about who takes responsibility for monitoring and measuring outcomes (Connolly et al, 2015;Connolly, 2016). Reid et al (2017) situated their study in the context of the knowledge brokerage literature on the basis that knowledge brokers aim to undertake strategies that draw together 'evidence producers' and 'evidence users' in order to maximise the opportunities for enabling synergies between knowledge and practice.…”

Contribution analysis as an approach to enable public managers to demonstrate public value

“…The contribution analysis process is made up of the following main components: i) Problem definition and model construction: The process starts with an analysis of the current situation being addressed -known as a 'situational analysis', or 'stakeholder analysis' (see Brugha and Varvasovszky, 2000), before identifying potential 'solutions'. The theory of change is articulated through a participatory and facilitated process within a workshop context (Connolly et al 2015). The main parties with an interest in the programme come together to discuss and define the problem including the intervention(s) required and how progress will be assessed.…”

“…The realities of planning processes which involve multiple and, frequently, cross-sectoral partners is such that ensuring cross-stakeholder buy-in throughout the contribution analysis process is challenging. A strategy to mitigate against conflict reaching dysfunctional levels (which could ultimately undermine and paralyse the process) is to have facilitators who, first, are knowledgeable about, and have experience of, the policy or programme contexts; and second, can fulfil the role of "critical friend" who can claim to have a level of independence from those who will be responsible for programme redesign, implementation and evaluation (see Connolly et al, 2015 who discuss how such workshops are conducted in terms of their specific contents and their structure with key reference to the learning and development literature).…”

“…Although evaluation has been the primary focus in studies of contribution analysis, of which most seem to talk about the approach rather than actually apply it (Delahais and Toulemonde, 2012), far less emphasis has been placed on how contribution analysis can support public managers to create value within partnership contexts (see Mayne, 2007Mayne, , 2012Wimbush et al, 2012;Connolly et al, 2015). This is a problem for public managers working in the modern public administration environment.…”

The use of inquiry-based learning in public administration education: Challenges and opportunities in the context of internationalization

“…1 However, to date there are no articles that bring together evidence on the variety of IBL approaches used to teach politics that might also be applicable to the subfield of public administration. The public administration discipline has tended to focus on how academics can use research methods to educate practitioners already working in the field through knowledge exchange (Ospina and Dodge, 2005;Connolly et al, 2015). Although many academics teaching public policy and administration undoubtedly make use of inquirybased approaches in elements of their teaching, there is need for greater conceptual clarity about what IBL means in this context, including the forms it might take, the challenges and opportunities of this approach, and how teachers using this approach might evaluate its effectiveness and engage in reflective practice to refine and modify their teaching.…”