The challenges of designing and piloting a system for measuring the impact of counselling provided by a national voluntary agency

Abstract

This paper describes the issues, processes and difficulties encountered and solutions proposed when introducing an evaluation system into a large and complex voluntary organisation providing counselling and relationship support services. The organisation Relate has 2,000 counsellors in over 600 locations operated by a network of 81 Relate Centres, and over 140,000 people use its services annually. The present project arose from a need clearly identified at a Mental Health Foundation conference in 1993, for a common measure to evaluate 'talking' therapies; to identify ways of measuring costs and benefits; and to assess the skills and competencies of therapists.This project set out to (1) identify and pilot a validated scale that could demonstrate the value of Relate; (2) develop and pilot a further measure of Relate's work, broader than the validated scale, in order to foster organisational learning and develop good practice. The pilot encountered a range of methodological challenges, and it is believed that recognition of these might smooth the way for any other similar organisation planning to develop an ongoing system for evaluation of counselling and relationship support.

Couple Counselling Outcomes in an Australian Not for Profit: Evidence for the Effectiveness of Couple Counselling Conducted Within Routine Practice

“…Others have noted the importance of consulting with staff to: (1) organise additional training; (2) provide a space to discuss their concerns (Aoun, Pennebaker & Janca, ; Crocker & Rissel, ); (3) prevent a study being perceived as threatening; d) overcome confidentiality issues; (4) plan implementation processes with them to overcome the challenges and to bolster their commitment (Callaly & Hallebone, ). The recommendations support findings of research undertaken in the relationship and family counselling sector in the UK and the US (Mecca, Rivera & Esposito, ; Mistral, Jackson, Brandling & McCarthy‐Young, ; Neuman, ; Ristau, ), particularly the importance of working with staff to prevent data collection adding a burdensome level of work (Mistral et al., ). Simple, practitioner and client friendly outcome measures are therefore typically employed in effectiveness studies.…”

Practitioners’ views and experiences of participating in a school counselling evaluation study

“…This was clearly communicated to practitioners throughout the process: they were aware that the Department of Education had provided funding for the evaluation and were represented on the Strategic Advisory Group. Mistral et al () reporting from their experience of implementing an outcome evaluation across a network of Relate counselling centres, also documented this as significant.…”

“…In doing so, it raises implications for the development of practice and outcome research within practice, in particular; the development of effective knowledge, attitude and skills based training to equip practitioners embrace outcome research. This is important if school counselling is to prove its legitimacy and secure strategic statutory funding and be accessible across all schools, to all children and young people who need support (Booth et al, ; Mistral et al, ; Pickstone et al, ).…”

“…The ongoing training and support provided through supervision, team meetings and the discussion board attempted to extend the effects of training into practice and was highly valued by staff in facilitating implementation. Mistral et al () also documented the significance of organisational support and resources to implementation.…”

“…Within the literature, and in particular that concerned with school counselling practice, little focus has been placed (with the exception of research reported by Mistral, Jackson, Brandling, & McCarthy‐Young,) on tracking the process of implementation of outcome research. Therefore not enough is known about the factors that are ‘vital for success’ or the potential ‘roadblocks’ that can delay or inhibit the process (Gotham, ).…”