Evaluating the use of locally-based health facility assessments in Afghanistan: a pilot study of a novel research method

Abstract

BackgroundThrough the Balanced Scorecard program there have been independent, annual and nationwide assessments of the Afghan health system from 2004 to 2013. During this period, Afghanistan remained in a dynamic state of conflict, requiring innovative approaches to health service evaluation in insecure areas. The primary objective of this pilot study was to evaluate the reliability of health facility assessments conducted by a novel, locally-based data collection method compared to a standard survey team.MethodsIn this cross-sectional study, one standard survey team of clinicians and multiple rapidly trained locally-based survey teams of teachers conducted health facility assessments in Badghis province, Afghanistan from March – August, 2010. Outpatient facilities covered under the country’s Basic Package of Health Services were eligible for inclusion. Both approaches attempted to survey as many health facilities as safely possible, up to 25 total facilities per method. Each facility assessed was scored on 23 health services indicators used to evaluate performance in the annual Balanced Scorecard national assessment. For facilities assessed by both survey methods, the indicator scores produced by each method were compared using Spearman’s correlation coefficients and linear regression analysis with generalized estimating equations.ResultsThe standard survey team was able to assess 11 facilities; the locally-based approach was able to assess these 11 facilities, as well as 13 additional facilities in areas of greater insecurity. Among the 11 facilities assessed by both approaches, 19 of 23 indicators were statistically similar by survey method (p < .05). Spearman’s coefficients varied widely from (−0.39) to (0.71). The differences were greatest for items requiring specialized data collector knowledge on reviewing patient records, patient examination and counseling, and health worker reported satisfaction.ConclusionsThis pilot study of a novel method of data collection in health facility assessments showed that an approach using locally-based survey teams provided markedly increased access to areas of insecurity. Though analysis was limited by small sample size, indicator scores used for facility evaluation were relatively comparable overall, but less reliable for items requiring clinical knowledge or when asking health worker opinions, suggesting that alternative approaches may be needed to assess these parameters in insecure environments.

A Systematic Review: The Dimensions utilized in the Performance Evaluation of Healthcare- An Implication during the COVID-19 Pandemic

“…HCWs' training-related KPIs, for example, were categorized under the learning and growth perspective in almost half of the resulted studies [30][31][32][48][49][50][56][57][58][59][64][65][66][67][68][69][70][74][75][76][77]. Meanwhile the rest of studies categorized them under the perspectives of HCWs [69,76], quality [71], capacity for service, provision/service capacity [30,31,50,68,69,75], and health care facility functionality [32]. These results are consistent with a study [5] that referred to the lack of defining measures and the lack of data standardization.…”

“…21 of the implementations were performed in hospitals (2ry and 3ry HCOs) [41,[49][50][51][52][53][54][55][56][57][58][59][60][61][62][63][64][65][66][67], and 15 in medical centers or health facilities (1ry HCOs) [30][31][32]48,[68][69][70][71][72][73][74][75][76].…”

Factors influencing performance by contracted non-state providers implementing a basic package of health services in Afghanistan

“…Outcome indicators are captured and measured separately by third party evaluators using the Balanced Score Card (BSC). The BSC has six domains [ 25 ]. (Additional file 1 provides information on BSC performance over time for each of the six provinces examined in this study).…”

Conflict, displacement and sexual and reproductive health services in Mali: analysis of 2013 health resources availability mapping system (HeRAMS) survey

“…Epidemiology-based activities, such as HeRAMS, conducted during and after an emergency event contribute to better understanding of the situation, needs and provide actionable information [ 5 ]. Therefore, it will be essential to keep improving the HeRAMS survey by examining and aligning the questions, indicators and their definitions, where possible, with other relevant and/or applicable tools such as Service availability and Readiness Assessment (SARA) and Balance Scorecard to facilitate the continuity and comparability of monitoring outside of emergencies [ 8 , 10 , 24 ].…”