What to expect when you're evaluating healthcare improvement: a concordat approach to managing collaboration and uncomfortable realities

Abstract

Evaluation of improvement initiatives in healthcare is essential to establishing whether interventions are effective and to understanding how and why they work in order to enable replication. Although valuable, evaluation is often complicated by tensions and friction between evaluators, implementers and other stakeholders. Drawing on the literature, we suggest that these tensions can arise from a lack of shared understanding of the goals of the evaluation; confusion about roles, relationships and responsibilities; data burdens; issues of data flows and confidentiality; the discomforts of being studied and the impact of disappointing or otherwise unwelcome results. We present a possible approach to managing these tensions involving the co-production and use of a concordat. We describe how we developed a concordat in the context of an evaluation of a complex patient safety improvement programme known as Safer Clinical Systems Phase 2. The concordat development process involved partners (evaluators, designers, funders and others) working together at the outset of the project to agree a set of principles to guide the conduct of the evaluation. We suggest that while the concordat is a useful resource for resolving conflicts that arise during evaluation, the process of producing it is perhaps even more important, helping to make explicit unspoken assumptions, clarify roles and responsibilities, build trust and establish open dialogue and shared understanding. The concordat we developed established some core principles that may be of value for others involved in evaluation to consider. But rather than seeing our document as a ready-made solution, there is a need for recognition of the value of the process of co-producing a locally agreed concordat in enabling partners in the evaluation to work together effectively.

The role of embedded research in quality improvement: a narrative review

“…One way to deal with the challenges connected with dual affiliation is to agree on clear guidelines from the beginning to manage expectations 50. The guidelines might define the role of the researcher, types of studies they will be able to undertake, study timeframes and feedback processes 14…”

Employing the arts for knowledge production and translation: Visualizing new possibilities for women speaking up about safety concerns in maternity

“…Team members’ personal belief in and commitment to the project helped with mediation of differences. These insights demonstrate that, for multiknowledge conglomeration or bricolage to occur, sufficient supportive infrastructures need to be in place, and mechanisms need to be set up early on in the project to foster open dialogue, trust and shared understandings (eg via a locally agreed concordat) …”

“…These insights demonstrate that, for multiknowledge conglomeration or bricolage 58 to occur, sufficient supportive infrastructures need to be in place, and mechanisms need to be set up early on in the project to foster open dialogue, trust and shared understandings (eg via a locally agreed concordat). 59 For those women that did participate, within the democratic tradition, involvement is seen as something that has intrinsic value in and of itself. 2 We employed a number of strategies to level the power differentials that accompany the co-production process.…”

What is the potential of patient shadowing as a patient-centred method?

“…Issues remain to be settled in relation to the status of shadowing as a relational method, what shadowers do, the role of the patient in the planning and conducting of shadowing and how to convert the findings into impacts on patients' care experiences. Selected insights from ethnography11 15 16 and current debates on coproduction17 18 could shed further light on how these challenges might optimally be addressed.…”

“…In public healthcare services tensions may emerge between the need to deliver better care to individual patients and the shrinking resources available. Ultimately, for patient shadowing to produce relevant knowledge, shadowers should be able and willing to provide feedback in actionable ways and to manage relationships effectively, especially when offering uncomfortable news 18. Yet this may be especially controversial for shadowers who are employed by the same organisation where they conduct shadowing.…”