“Leveling up” for change: Application of a multilevel system capacity change model to evaluate U.S. Department of Defense global health engagement activities

Abstract

Evaluating within complex systems is challenging because of how complexity affects the identification and observation of outcomes. U.S. Department of Defense (U.S. DoD) capacity building global health engagements are often difficult to measure due to the conflation of levels of analysis and confounding variables, hindering the explanation of change effects. This article will illustrate two case examples where a boundary-driven systems framework was utilized to integrate systems thinking into U.S. DoD capacity building programs and associated evaluations. The findings from the first case led to developing a theory of change that was later tested and refined in the second case to establish the multilevel system (MLS) concept model. Based on these findings, the four distinct system boundaries and subcomponents of the MLS concept model were refined to include changes within the organizational system. The development of the MLS model allowed for the explicit framing of efforts, measurement and analysis, and the alignment of program activities and observed outcomes; while still allowing for the illumination of emergent change effects in a complex system.