Comparison of Weighting Methods to Understand Improved Outcomes Attributable to Public Health Nursing Interventions

Abstract

Background The complex work of public health nurses (PHNs) specifically related to mental health assessment, intervention, and outcomes, makes it difficult to quantify and evaluate the improvement in client outcomes attributable to their interventions. Objectives We examined heterogeneity across parents of infants served by PHNs receiving different interventions; compared the ability of traditional propensity scoring methods versus energy balancing weight techniques to adjust for the complex and stark differences in baseline characteristics among those receiving different interventions; and evaluated the causal effects of the quantity and variety of PHN interventions on client health and social outcomes. Methods This retrospective study of 4,109 clients used existing Omaha System data generated during the routine documentation of PHN home visit data. We estimated the effects of intervention by computing and comparing weighted averages of the outcomes within the different treatment groups using two weighting methods: (a) inverse probability of treatment (propensity score) weighting and (b) energy balancing weights (EBWs). Results Clients served by PHNs differed in baseline characteristics with clients with more signs/symptoms. Both weighting methods reduced heterogeneity in the sample. EBWs were more effective than inverse probability of treatment weighting in adjusting for multifaceted confounding and resulted in close balance of 105 baseline characteristics. Weighting the sample changed outcome patterns, especially when using energy-balancing weights. Clients who received more PHN interventions and a wider variety of them had improved knowledge, behavior, and status outcomes with no plateau over time, whereas the unweighted sample showed plateaus in outcomes over the course of home visiting services. Discussion Causal analysis of PHN-generated data demonstrated PHN intervention effectiveness for clients with mental health signs/symptoms. EBWs are a promising tool for evaluating the true causal effect of PHN home-visiting interventions.