Valuing the impact of health and social care programs using social return on investment analysis: how have academics advanced the methodology? A systematic review

Abstract

ObjectivesTo identify how social return on investment (SROI) analysis—traditionally used by business consultants—has been interpreted, used and innovated by academics in the health and social care sector and to assess the quality of peer-reviewed SROI studies in this sector.DesignSystematic review.SettingsCommunity and residential settings.ParticipantsA wide range of demographic groups and age groups.ResultsThe following databases were searched: Web of Science, Scopus, CINAHL, Econlit, Medline, PsychINFO, Embase, Emerald, Social Care Online and the National Institute for Health and Care Excellence. Limited uptake of SROI methodology by academics was found in the health and social care sector. From 868 papers screened, 8 studies met the criteria for inclusion in this systematic review. Study quality was found to be highly variable, ranging from 38% to 90% based on scores from a purpose-designed quality assessment tool. In general, relatively high consistency and clarity was observed in the reporting of the research question, reasons for using this methodology and justifying the need for the study. However, weaknesses were observed in other areas including justifying stakeholders, reporting sample sizes, undertaking sensitivity analysis and reporting unexpected or negative outcomes. Most papers cited links to additional materials to aid in reporting. There was little evidence that academics had innovated or advanced the methodology beyond that outlined in a much-cited SROI guide.ConclusionAcademics have thus far been slow to adopt SROI methodology in the evaluation of health and social care interventions, and there is little evidence of innovation and development of the methodology. The word count requirements of peer-reviewed journals may make it difficult for authors to be fully transparent about the details of their studies, potentially impacting the quality of reporting in those studies published in these journals.PROSPERO registration numberCRD42018080195.

Social return on investment (SROI) method to evaluate physical activity and sport interventions: a systematic review

“…Conversely, a weakness of the field is that none of the studies included negative outcomes. This weakness has also been noted in previous SROI reviews [15,23].…”

“…The next step is attributing values to the outcomes, referred to as the monetisation. Monetising the social outcomes of an intervention is one of the main challenges of the method as some may be difficult to monetise (e.g., subjective wellbeing or improved self-esteem) [16,23]. To do so, financial proxies are used: they provide an estimate of financial value for outcomes or benefits that have no market value.…”

Health Impact and Social Value of Interventions, Services, and Policies: A Methodological Discussion of Health Impact Assessment and Social Return on Investment Methodologies

“…These examples may reflect that the two methodologies discussed in this paper can be applied together on interventions or services, as opposed to assessing a wide range of impact and social value of strategic initiatives and policies. In addition, the lack of academic literature found in this scoping review is reflective of other reviews which note the little uptake of SROI methodology by academics (16,20). This could potentially be due to lack of resources to use SROI methodology, or a lack of awareness or understanding by practitioners and policy makers of the methodology.…”

“…This could ensure a wide range of potential outcomes and indicators are covered, including vulnerable groups, inequities and inequalities. SROI was also noted to potentially privilege stakeholder perspectives over other types of evidence (20) which could be avoided by triangulation used within HIAs to consider and compare different available evidence.…”

“…Vice-versa, stakeholder engagement methodologies outlined in the SROI process have been described as vague in their approach, particularly in relation to stakeholder participation, and have been claimed to prioritize stakeholder evidence over other types (19,20). In addition, HIA can complement the SROI method in understanding how and why impacts occur, which can help decide which outcomes would be more important or useful to valuate (21).…”