Reliability of the Service Trip Audit Tool to assess the quality of short-term medical missions

Abstract

Background We aimed to assess the adherence of short-term medical missions (STMMs) operating in Latin America and the Caribbean (LAC) to key best practices using the Service Trip Audit Tool (STAT) and to calculate the inter-rater reliability of the data points. This tool was based on a previously published inventory of 18 STMM best practices. Methods Programme administrators and recent volunteers from 335 North American organizations offering STMMs in LAC were invited to complete the STAT anonymously online. Adherence to each of 18 best practices was reported as either ‘yes’, ‘no’ or ‘not sure’. Fleiss’ κ was used to assess inter-rater agreement of the responses. Results A total of 194 individuals from 102 organizations completed the STAT (response rate 30.4%; 102/335 organizations) between 12 July and 7 August 2017. Reported adherence was >80% for 9 of 18 best practices. For 37 non-governmental organizations (NGOs) with multiple raters, inter-rater agreement was moderate to substantial (κ>0.4) for 12 of 18 best practices. Conclusions This is the first study to evaluate adherence to STMM best practices. Such an objective evaluation will be valuable to governments, volunteers and NGO donors who have an interest in identifying high-quality partners. Assessment and monitoring of STMMs through self-audit may be foundational steps towards quality improvement.

Capacity Building During Short‐Term Surgical Outreach Trips: A Review of What Guidelines Exist

“…and direct all trip activities by local community needs 11 [ 27 – 29 , 38 , 41 – 43 , 46 , 47 , 50 ] Ensure appropriate outcomes, including measures of quality, are collected during trips and reviewed afterward 9 [ 8 , 33 , 35 , 39 , 41 , 43 , 45 , 48 , 50 ] Maintain an internal registry or database that is easily accessible and understood by all providers to collect patient outcomes 4 [ 39 , 42 , 47 , 55 ] Use a safe surgery checklist during outreach 4 [ 32 , 33 , 35 , 45 ] Ensure continuity of care by having a local clinician involved in all steps of care 3 [ 26 , 33 , 54 ] For long-term sustainability of impact, have successive trips in one location and focus on health systems strengthening to promote independence of health system 5 [ 8 , 33 , 48 , 50 , 58 ] Embed trips within community-led effo...…”

“…The most frequently classified domain was Community Impact (from 23 (66%) studies) and the least frequent was Finance, with components from nine (25%) studies [28, 29, 35, 41, 48, 49, 55, 57, 58].…”

“…Guideline elements f Sources Partnerships Establishment of bidirectional partnerships that are collaborative are critical 6 [29,32,41,48,52,57] Short-term trips should be a component of long-term partnerships to mitigate adverse consequences, ensure partnerships do not end abruptly, and to ensure sustainability 5 [28,29,33,41,57] Engage with partners outside of the local hospital, such as public health officials, community health workers, local pharmacies, local NGOs, or local government agencies 11 [28,33,41,46,50,53,[55][56][57][58] Foster coalitions between outreach organizations to streamline care 2 [42,47] Professional development Activities should align with the goals of the public sector, follow local patient protection practices, and meet local quality standards 6 [32,33,36,48,50,56] Do not practice outside of scope, which is especially important when trainees (including students, residents, and fellows) are assisting during the trip 11 [26, 28-30, 33, 36, 39, 43, 50, 53] Follow guidelines for using any expired or reused supplies, leave any equipment necessary for follow-up care or future planned procedures with the local hospital 3 [28,30,32] Have policies in place for the informed consent and media collection process, including obtaining patient consent for any photography and research 5 [28,33,39,42,45] Comm...…”

“…Some outreach trips have proposed different methods of building financial capacity in surgical systems to assist in bridging the initial high cost of surgical scale-up, such as a sliding scale patient payment system to financially support trips and allow for funds to be allocated to larger health projects [48] or calculating and reimbursing all costs to the local community during the trip [22]. We recommend continued investigation into and incorporation of sustainable financial There is an established communication system between local physicians and the organization 1 [39] Pre-trip planning should include communication of pre-selected patients (if desired), the infrastructure to provide care, risk factors, and conditions needed to treat 6 [28,30,34,36,37,39,59] Collaborations with other organizations, such as local NGOs or other international outreach organizations, is recommended 4 [30,34,43,44] The volunteer team has a formal staffing plan and should be multidisciplinary 6 [41,43,47,50,55] Ensure there is a plan for referrals and follow-up 10 [28,30,36,38,39,41,47,50,55] Have post-trip debriefing and solicit feedback from multiple stakeholders to reflect and plan next trips programs and models during short-term surgical outreach trips. While many efforts to include capacity building during outreach trips were present in this study, actionable steps were noticeably missing.…”

“…Guideline elements f Sources Partnerships Establishment of bidirectional partnerships that are collaborative are critical 6 [ 29,32,41,48,52,57] Short-term trips should be a component of long-term partnerships to mitigate adverse consequences, ensure partnerships do not end abruptly, and to ensure sustainability 5 [ 28,29,33,41,57] Engage with partners outside of the local hospital, such as public health officials, community health workers, local pharmacies, local NGOs, or local government agencies 11 [28,33,41,46,50,53,[55][56][57][58] Foster coalitions between outreach organizations to streamline care 2 [42,47] Professional development…”