Designing Difference in Difference Studies: Best Practices for Public Health Policy Research

Wing, Coady; Simon, Kosali; Bello-Gómez, Ricardo A.

doi:10.1146/annurev-publhealth-040617-013507

Cited by 1,065 publications

(910 citation statements)

References 105 publications

Supporting

Mentioning

897

Contrasting

Unclassified

Order By: Relevance

“…The results reveal a pattern that is similar to the preferred estimates below, and show that using time fixed effects does not statistically lead to a higher log‐likelihood value, compared to a linear trend. Consistent with the ITS and CITS exercises, the use of alternative methods to account for non‐stationarity and autocorrelation reinforces the conclusions of this paper, illustrating their robustness …”

Section: Methodssupporting

confidence: 79%

“…Table A.2 shows that DD is preferred to CITS; the diverging trend hypothesis does not hold statistically, leading to overfitting under CITS . An explanation why CITS performs less well here compared to other studies is perhaps the fact that the number of pre‐intervention data points in this study is too small (five data points: March‐July 2010) to properly establish pre‐intervention trends and observe trend divergence …”

Section: Methodsmentioning

confidence: 84%

“…The second method is comparative/controlled interrupted time series (CITS). The method has been proved to be valid and precise, and is considered more flexible than DD, rendering it superior under certain conditions. More specifically, while the DD method assumes that outcome trends are similar, that is, “common,” between the intervention and the non‐intervention groups, the CITS method relaxes the assumption and allows those trends to diverge .…”

Section: Methodsmentioning

confidence: 99%

See 2 more Smart Citations

Effectiveness of antimalarial interventions in Nigeria: Evidence from facility‐level longitudinal data

Witvorapong

Yakubu²

2019

Health Services Research

View full text Add to dashboard Cite

Objective To evaluate the effectiveness of a program of antimalarial interventions implemented in 2010‐2013 in Niger State, Nigeria. Data Sources Utilization reports from 99 intervention and 51 non‐intervention health facilities from the Niger State Malaria Elimination Program, supplemented by data on facility‐level characteristics from the Niger State Primary Health Care Development Agency and Local Government Malaria Control units. Study Design Estimated with mixed‐effects negative binomial modeling, a difference‐in‐differences method was used to quantify the impact of the program on the number of febrile illness cases and confirmed malaria cases. Potential confounding factors, non‐stationarity, seasonality, and autocorrelation were explicitly accounted for. Data Extraction Methods Data were retrieved from hard copies of utilization reports and manually inputted to create a panel of 5550 facility‐month observations. Principal Findings The program was implemented in two phases. The first phase (August 2010‐June 2012) involved the provision of free artemisinin‐based combination therapies, long‐lasting insecticidal nets, and intermittent preventive treatments. In the second phase (July 2012‐March 2013), the program introduced an additional intervention: free parasite‐based rapid diagnostic tests. Compared to the pre‐intervention period, the average number of monthly febrile illness and malaria cases increased by 20.876 (P < 0.01) and 22.835 (P < 0.01) in the first phase, and by 19.007 (P < 0.05) and 19.681 (P < 0.05) in the second phase, respectively. The results are consistent across different evaluation methods. Conclusions This study suggests that user‐fee removal leads to increased utilization of antimalarial services. It motivates future studies to cautiously select their investigative methods.

show abstract

Section: Methodssupporting

confidence: 79%

Section: Methodsmentioning

confidence: 84%

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Effectiveness of antimalarial interventions in Nigeria: Evidence from facility‐level longitudinal data

Witvorapong

Yakubu²

2019

Health Services Research

View full text Add to dashboard Cite

show abstract

“…The cross‐sectional observations in the panel consist of the 197 most populous cities in the United States in 1990 and all other places (ie, localities other than those cities) aggregated together within each state (Table ). We employed a difference‐in‐differences approach (sometimes called a 2‐way fixed effects model), which is commonly used for identifying treatment effects when there is staggered policy implementation . As used here, this model compares the difference in homicide rates in a city before and after the implementation of a firearm law in that state to contemporaneous changes in homicide rates in cities located in states without such laws.…”

Section: Methodsmentioning

confidence: 99%

The Impact of State Firearm Laws on Homicide Rates in Suburban and Rural Areas Compared to Large Cities in the United States, 1991‐2016

Siegel

Solomon

Knopov

et al. 2019

The Journal of Rural Health

View full text Add to dashboard Cite

Purpose This article aims to examine whether state firearm laws impact homicide rates differently in suburban and rural areas compared to large cities in the United States. Methods We analyzed serial, cross‐sectional data for the 26‐year period 1991‐2016 using a panel design. We examined the relationship between 6 specific state firearm laws and homicide rates in large cities (those with greater than 100,000 people in 1990) and in all geographic areas outside of these cities. Using a city‐level fixed effects negative binomial regression, we modeled the number of homicides as a function of state firearm laws, while controlling for time fixed effects and time‐varying state‐ and city‐level sociodemographic factors. Findings Two policies—universal background checks and “may issue” laws that required a heightened showing of suitability for concealed carry—were associated with lower firearm homicide rates in large cities but were not associated with firearm homicide rates in suburban and rural areas. In contrast, laws that prohibited gun possession by people convicted of a violent misdemeanor were associated with lower firearm homicide rates in suburban and rural areas, but were not associated with firearm homicide rates in large cities. Permit requirements were associated with lower firearm homicide rates in both large cities and suburban and rural areas. Conclusions This article provides the first evidence that state firearm laws may have a differential impact on firearm homicide rates in suburban and rural areas compared to urban areas in the United States.

show abstract

“…In Table , we used difference‐in‐differences regression analyses to evaluate the effects of screening mammography on breast cancer incidence and mortality rates. This method has been rigorously developed in the causal inference literature, but its roots date back to much earlier use in public health, including in John Snow's famous studies of cholera transmission in 1850’s London . In the difference‐in‐differences method, the effect of an intervention is evaluated observationally by comparing the changes in outcomes between groups that receive the intervention and control groups that do not.…”

Section: Methodsmentioning

confidence: 99%

Long‐term relationships between screening rates, breast cancer characteristics, and overdiagnosis in US counties, 1975–2009

Harding

Pompei

Burmistrov

et al. 2018

Intl Journal of Cancer

View full text Add to dashboard Cite

Effects of mammography screening in the general population are disputed. Screening rates differ greatly between US counties, providing a natural opportunity to investigate effects of screening. We compared mammography screening rates with the types and outcomes of breast cancers diagnosed in US counties. The county screening rate was defined as the proportion of women age ≥40 with ≥1 mammogram in the past 2 years (range, 34–91%). Two periods were analyzed: 1975–2009 (612,941 breast cancer cases, 195 counties) and 1996–2009 (645,057 cases, 211–547 counties). Multiple signs of overdiagnosis were observed: First, breast cancer incidence increased as screening became common. Second, incidence stopped increasing once screening rates stabilized. Third, the increases in incidence were limited to age groups receiving screening. Fourth, the increases were larger in counties where screening became more common. Fifth, the increases were limited to small and early‐stage breast cancers (which are consistent with overdiagnosis). Sixth, compensatory reductions in large and advanced‐stage breast cancers were much smaller than the increases. Difference‐in‐differences regression analysis suggested 31% (95% CI: 28–34%) of breast cancers diagnosed in 1996–2009 were overdiagnosed. Screening rates correlated with increased incidence for all hormone receptor statuses, HER2 statuses, and grades. Reductions in breast cancer mortality during 1975–2009 were similar in screened and unscreened age groups. Overall, we found repeated signs that breast cancer overdiagnosis is widespread in the US, but the biological nature of overdiagnosed tumors remains unclear. Mortality benefits of screening, though they may be present and substantial, could not be detected at the population level.

show abstract

Designing Difference in Difference Studies: Best Practices for Public Health Policy Research

Cited by 1,065 publications

References 105 publications

Effectiveness of antimalarial interventions in Nigeria: Evidence from facility‐level longitudinal data

Effectiveness of antimalarial interventions in Nigeria: Evidence from facility‐level longitudinal data

The Impact of State Firearm Laws on Homicide Rates in Suburban and Rural Areas Compared to Large Cities in the United States, 1991‐2016

Long‐term relationships between screening rates, breast cancer characteristics, and overdiagnosis in US counties, 1975–2009

Contact Info

Product

Resources

About