Well‐Balanced or too Matchy–Matchy? The Controversy over Matching in Difference‐in‐Differences

Ryan, Andrew M.

doi:10.1111/1475-6773.13015

Cited by 17 publications

(14 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The difference-in-differences method then controls for unobserved but fixed omitted variables, relying on the assumption that the counterfactual trends in the treatment and control groups are the same. The matching procedure makes this assumption of parallel trends more plausible by ensuring that the outcomes in the treatment and control groups are similar in levels at baseline [ 34 – 36 ]. Another useful property of matching is that it reduces bias from the potential misspecification of the subsequent regression model [ 37 ].…”

Section: Methodsmentioning

confidence: 99%

Performance bonuses and the quality of primary health care delivered by family health teams in Brazil: A difference-in-differences analysis

et al. 2022

View full text Add to dashboard Cite

Background Pay-for-performance (P4P) programmes to incentivise health providers to improve quality of care have been widely implemented globally. Despite intuitive appeal, evidence on the effectiveness of P4P is mixed, potentially due to differences in how schemes are designed. We exploited municipality variation in the design features of Brazil’s National Programme for Improving Primary Care Access and Quality (PMAQ) to examine whether performance bonuses given to family health team workers were associated with changes in the quality of care and whether the size of bonus mattered. Methods and findings For this quasi-experimental study, we used a difference-in-differences approach combined with matching. We compared changes over time in the quality of care delivered by family health teams between (bonus) municipalities that chose to use some or all of the PMAQ money to provide performance-related bonuses to team workers with (nonbonus) municipalities that invested the funds using traditional input-based budgets. The primary outcome was the PMAQ score, a quality of care index on a scale of 0 to 100, based on several hundred indicators (ranging from 598 to 660) of health care delivery. We did one-to-one matching of bonus municipalities to nonbonus municipalities based on baseline demographic and economic characteristics. On the matched sample, we used ordinary least squares regression to estimate the association of any bonus and size of bonus with the prepost change over time (between November 2011 and October 2015) in the PMAQ score. We performed subgroup analyses with respect to the local area income of the family health team. The matched analytical sample comprised 2,346 municipalities (1,173 nonbonus municipalities; 1,173 bonus municipalities), containing 10,275 family health teams that participated in PMAQ from the outset. Bonus municipalities were associated with a 4.6 (95% CI: 2.7 to 6.4; p < 0.001) percentage point increase in the PMAQ score compared with nonbonus municipalities. The association with quality of care increased with the size of bonus: the largest bonus group saw an improvement of 8.2 percentage points (95% CI: 6.2 to 10.2; p < 0.001) compared with the control. The subgroup analysis showed that the observed improvement in performance was most pronounced in the poorest two-fifths of localities. The limitations of the study include the potential for bias from unmeasured time-varying confounding and the fact that the PMAQ score has not been validated as a measure of quality of care. Conclusions Performance bonuses to family health team workers compared with traditional input-based budgets were associated with an improvement in the quality of care.

show abstract

Section: Methodsmentioning

confidence: 99%

Performance bonuses and the quality of primary health care delivered by family health teams in Brazil: A difference-in-differences analysis

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Another important question relates to whether researchers should condition on pretreatment outcomes. Proponents of including pre-treatment outcomes argue that controlling for lagged outcomes can reduce bias from unobserved confounders (Ryan, 2018). 10 However, conditioning on lagged outcomes need not necessarily reduce bias.…”

Section: Standard Linear Regressionmentioning

confidence: 99%

What's Trending in Difference-in-Differences? A Synthesis of the Recent Econometrics Literature

Roth¹,

Sant’Anna²,

Bilinski³

et al. 2022

Preprint

View full text Add to dashboard Cite

This paper synthesizes recent advances in the econometrics of difference-in-differences (DiD) and provides concrete recommendations for practitioners. We begin by articulating a simple set of "canonical" assumptions under which the econometrics of DiD are well-understood. We then argue that recent advances in DiD methods can be broadly classified as relaxing some components of the canonical DiD setup, with a focus on piq multiple periods and variation in treatment timing, piiq potential violations of parallel trends, or piiiq alternative frameworks for inference. Our discussion highlights the different ways that the DiD literature has advanced beyond the canonical model, and helps to clarify when each of the papers will be relevant for empirical work. We conclude by discussing some promising areas for future research.

show abstract

“…The difference-in-differences method then controls for unobserved but fixed omitted variables, relying on the assumption that the counterfactual trends in the treatment and control groups are the same. The matching procedure makes this assumption of parallel trends more plausible by ensuring that the outcomes in the treatment and control groups are similar in levels at baseline [34][35][36]. Another useful property of matching is that it reduces bias from the potential misspecification of the subsequent regression model [37].…”

Section: Study Setting and Designmentioning

confidence: 99%

Outcomes following SARS-CoV-2 infection in individuals with and without solid organ transplantation—A Danish nationwide cohort study

Overvad

Koch

Jespersen

et al. 2022

American Journal of Transplantation

View full text Add to dashboard Cite

The risk of severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2) infection, hospitalization and death, and the effects of SARS‐CoV‐2 vaccines in solid organ transplant recipients (SOTRs) is still debated. We performed a nationwide, population‐based, matched cohort study, including all Danish SOTRs ( n = 5184) and a matched cohort from the general population ( n = 41 472). Cox regression analyses were used to calculate incidence rate ratios (IRRs). SOTRs had a slightly increased risk of SARS‐CoV‐2 infection and were vaccinated earlier than the general population. The overall risk of hospital contact with COVID‐19, severe COVID‐19, need for assisted respiration, and hospitalization followed by death was substantially higher in SOTRs (IRR: 32.8 95%CI [29.0–37.0], 9.2 [6.7–12.7], 12.5 [7.6–20.8], 12.4 [7.9–12.7]). The risk of hospitalization and death after SARS‐CoV‐2 infection decreased substantially in SOTRs after the emergence of the Omicron variant (IRR: 0.45 [0.37–0.56], 0.17 [0.09–0.30]). Three vaccinations reduced the risk of SARS‐CoV‐2 infection only marginally compared to two vaccinations, but SOTRs with three vaccinations had a lower risk of death (IRR: 022 [0.16–0.35]). We conclude that SOTRs have a risk of SARS‐CoV‐2 infection comparable to the general population, but substantially increased the risk of hospitalization and death following SARS‐CoV‐2 infection. A third vaccination only reduces the risk of SARS‐CoV2 infection marginally, but SOTRs vaccinated 3 times have reduced mortality.

show abstract

Well‐Balanced or too Matchy–Matchy? The Controversy over Matching in Difference‐in‐Differences

Cited by 17 publications

References 15 publications

Performance bonuses and the quality of primary health care delivered by family health teams in Brazil: A difference-in-differences analysis

Performance bonuses and the quality of primary health care delivered by family health teams in Brazil: A difference-in-differences analysis

What's Trending in Difference-in-Differences? A Synthesis of the Recent Econometrics Literature

Outcomes following SARS-CoV-2 infection in individuals with and without solid organ transplantation—A Danish nationwide cohort study

Contact Info

Product

Resources

About