Estimating the impact of nutrition and physical activity policies with quasi-experimental methods and simulation modelling: an integrative review of methods, challenges and synergies

Abstract

Background The promotion of healthy lifestyles has high priority on the global public health agenda. Evidence on the real-world (cost-)effectiveness of policies addressing nutrition and physical activity is needed. To estimate short-term policy impacts, quasi-experimental methods using observational data are useful, while simulation models can estimate long-term impacts. We review the methods, challenges and potential synergies of both approaches for the evaluation of nutrition and physical activity policies. Methods We performed an integrative review applying purposive literature sampling techniques to synthesize original articles, systematic reviews and lessons learned from public international workshops conducted within the European Union Policy Evaluation Network. Results We highlight data requirements for policy evaluations, discuss the distinct assumptions of instrumental variable, difference-in-difference, and regression discontinuity designs and describe the necessary robustness and falsification analyses to test them. Further, we summarize the specific assumptions of comparative risk assessment and Markov state-transition simulation models, including their extension to microsimulation. We describe the advantages and limitations of these modelling approaches and discuss future directions, such as the adequate consideration of heterogeneous policy responses. Finally, we highlight how quasi-experimental and simulation modelling methods can be integrated into an evidence cycle for policy evaluation. Conclusions Assumptions of quasi-experimental and simulation modelling methods in policy evaluations should be credible, rigorously tested and transparently communicated. Both approaches can be applied synergistically within a coherent framework to compare policy implementation scenarios and improve the estimation of nutrition and physical activity policy impacts, including their distribution across population sub-groups.

Caloric reductions needed to achieve obesity goals in Mexico for 2030 and 2040: A modeling study

“…Several limitations need to be acknowledged. Microsimulation modeling has several advantages and limitations that have been addressed in more depth elsewhere [ 53 ]. In the microsimulation model, we assumed that physical activity would be constant over time and the only source of imbalance was the energy intake through the projected MEGs from 2018 to 2040.…”

Projected health and economic impacts of sugar-sweetened beverage taxation in Germany: A cross-validation modelling study

“…To further improve the accuracy of population health modelling studies and their relevance for policy makers generally and specifically in Germany many avenues for future research exist. First, the quantification of causal effects of the cross-sectoral, multidimensional, and heterogenous behavioural response to health and social policies is of particular importance [9]. This will both, improve our understanding of policy mechanisms and lead to better projections from policy simulations.…”

“…A tiered structure here means that tax levels are differentiated based on beverage sugar content using predefined thresholds (e.g., 0.18 British pound sterling [₤] per litre for drinks containing 5 to 8 grams [g] sugar per 100 millilitre [ml] and ₤0.24 per litre for drinks containing more than 8 g sugar per 100 ml in case of the Soft Drinks Industry Levy [SDIL] in the United Kingdom [UK]). However, although early observational studies indicate that SSB taxes could be effective in preventing obesity, generating robust empirical evidence on their impact on long-term health and economic outcomes is difficult [8,9].…”

Dietary behaviour and physical activity policies in Europe: learnings from the Policy Evaluation Network (PEN)

“…Thus, we propose to synergistically combine both approaches to overcome their limitations. 34 However, assumptions behind models must be transparent and credible. This implies rigorous testing and validation through recognized robustness/sensitivity checks.…”

“…Within PEN we conducted an integrative review on the methods, challenges and potential synergies of both approaches for the evaluation of nutrition and physical activity policies. 34 The application of these methods for policy evaluation increased substantially over the last decade. It is important that the assumptions of quasi-experimental and simulation modelling methods should be credible and rigorously tested and transparently communicated.…”