Propensity Score Matching in Accounting Research

Abstract

Propensity score matching (PSM) has become a popular technique for estimating average treatment effects (ATEs) in accounting research. In this study, we discuss the usefulness and limitations of PSM relative to more traditional multiple regression (MR) analysis. We discuss several PSM design choices and review the use of PSM in 86 articles in leading accounting journals from 2008–2014. We document a significant increase in the use of PSM from zero studies in 2008 to 26 studies in 2014. However, studies often oversell the capabilities of PSM, fail to disclose important design choices, and/or implement PSM in a theoretically inconsistent manner. We then empirically illustrate complications associated with PSM in three accounting research settings. We first demonstrate that when the treatment is not binary, PSM tends to confine analyses to a subsample of observations where the effect size is likely to be smallest. We also show that seemingly innocuous design choices greatly influence sample composition and estimates of the ATE. We conclude with suggestions for future research considering the use of matching methods. Data Availability: All data used are available from sources cited in the text.

Is the SEC captured? Evidence from comment-letter reviews

“…We then match to each PC firm a non-PC firm from the same year, reviewed by the same SEC office, and of similar characteristics along all of the covariates used in Table 3, using a propensity score matching method. We match PC and non-PC firms within a predefined propensity score radius (or "caliper") of 0.0005, and allow for replacement in the selection of matches to ensure that we find a meaningful match for each of the PC firms (Shipman et al 2017). 7 For the 4,301 PC firm-years defined as having non-zero PAC contributions, we find matches for 768 PC firm-years yielding a total sample size of 1,536.…”

Fool's gold or value for money? The link between abnormal audit fees, audit firm type, fair‐value disclosures, and market valuation

“…However, the functional form of the underlying relation between auditor choice and these variables is unknown, and so it is unclear whether the inclusion of control variables sufficiently captures these differences (Shipman, Swanquist, & Whited, ). Therefore, we apply propensity score matching, which has been used before to account for differences between Big 4 and non‐Big 4 clients (e.g., DeFond, Erkens, & Zhang, ; Lawrence, Minutti‐Meza, & Zhang, ; Shipman et al, ). This means that we match each bank with a Big 4 auditor to the bank with a non‐Big 4 auditor that most closely resembles this bank.…”

The impact of the Sarbanes–Oxley Section 404(b) exemption on earnings informativeness

“…However, the use of this approach in accounting studies has been the subject of recent criticism in the literature (Chen, Hribar, & Melessa, 2018;Lennox, Francis, & Wang, 2012;Shipman, Swanquist, & Whited, 2017). This gives rise to a potential endogeneity problem with respect to uncontrolled differences between the two sets of companies.…”

“…This would indicate the companies would be inclined to keep complying with Section 404(b) voluntarily. 12 As an additional analysis, we use propensity score matching following the approach in Shipman et al (2017). 51 observations that continue voluntary compliance with SOX 404(b) beyond 1 year.…”