Validation of a novel tool for assessing newborn resuscitation skills among birth attendants trained by the Helping Babies Breathe program

Abstract

Objective To validate a simplified objective structured clinical examination (OSCE) tool for evaluating the competency of birth attendants in low‐resource countries who have been trained in neonatal resuscitation by the Helping Babies Breathe (HBB) program. Methods A prospective cross‐sectional study of the OSCE tool was conducted among trained birth attendants working at dispensaries, health centers, or hospitals in five regions of Tanzania between October 1, 2013, and May 1, 2014. A 13‐item checklist was used to assess clinical competency in a simulated newborn resuscitation scenario. The OSCE tool was simultaneously administered by HBB trainers and experienced external evaluators. Paired results were compared using the Cohen κ value to measure inter‐rater reliability. Participant performance was rated by health cadre, region, and facility type. Results Inter‐rater reliability was moderate (κ = 0.41–0.60) or substantial (κ = 0.61–0.80) for eight of the OSCE items; agreement was fair (κ = 0.21–0.41) for the remaining five items. The best OSCE performances were recorded among nurses and providers from facilities with high annual birth volumes. Conclusion The simplified OSCE tool could facilitate efficient implementation of national‐level HBB programs. Limitations in inter‐rater reliability might be improved through additional training.

Retention and use of newborn resuscitation skills following a series of helping babies breathe trainings for midwives in rural Ghana

“…The most effective method to refresh these skills, however, has yet to be determined. Several studies have documented either improvement on simulation exercise or in observed delivery room performance when students were provided with brief, frequent refresher training sessions, such as supervised daily practice [3], monthly 40 minute sessions [6], or visits to designated health centers every two weeks [4]. …”

“…The program includes a validated simulation evaluation tool called an Objective Structured Clinical Examination (OSCE) [3,4]. Several studies have demonstrated trainee improvement on the examination using a pre-post design [5–8].…”

Newborn Resuscitation Training in Resource-Limited Settings: A Systematic Literature Review

“…Ten (22.2%) of the 45 studies reported a breakdown of scores according to cadre, 26,36,39,40,44,50,53,55,56,74 and 7 (70.0%) of these studies showed that birth attendants from lower cadres (eg, midwives) improved significantly more from pretraining to posttraining than those from higher cadres (eg, physicians). 26,40,42,50,53,56,74 In all 7 of these studies, significant differences in pretraining test performance between different cadres were diminished or completely eliminated upon retesting after training.…”

“…Similarly, a study from HBB in Tanzania found that nurses outperformed health care providers of both higher and lower training levels on a standardized OSCE. 39 Two studies analyzed a breakdown of test scores based on the type of birth facility in which birth attendants work and both found a significant association between working in a facility with higher birth volume and higher scores on examinations. 39,53 Another provider-level factor whose association with the adoption and retention of skills has been explored is provider self-efficacy (ie, one's belief in their own competence).…”

“…39 Two studies analyzed a breakdown of test scores based on the type of birth facility in which birth attendants work and both found a significant association between working in a facility with higher birth volume and higher scores on examinations. 39,53 Another provider-level factor whose association with the adoption and retention of skills has been explored is provider self-efficacy (ie, one's belief in their own competence). Four studies (8.8%) measured selfefficacy by using questionnaires, and although all 4 studies reported increased self-efficacy of birth attendants with training, they were mixed on the value and relevance of self-efficacy.…”