Publication Bias and the Cross-Section of Stock Returns

Abstract

We develop an estimator for publication bias-adjusted returns and apply it to 156 published long-short portfolios. Our adjustment uses only in-sample data and provides sharper inferences than out-of-sample tests. Bias-adjusted returns are only 12.3% smaller than in-sample returns with a standard error of 1.7 percentage points. The small bias comes from the dispersion of returns across predictors, which is too large to be explained by data-mined noise. The bias is much smaller than post-publication decay (p-value ¡.0001), suggesting mispricing is important. Our results offer a different perspective about recent papers that find most published predictors are likely false. Authors have furnished an Internet Appendix, which is available on the Oxford University Press Web site next to the link to the final published paper online.

The Limits of p‐Hacking: Some Thought Experiments

“…My models can be thought of as variants of HLZ and CGS, which assume that risk premiums and alphas are all zero, and that the empirical tail is generated entirely through selection bias. Other papers that are based on this framework include Chen (2019) and Chen and Zimmermann (2020b), who find quantitatively similar point estimates.…”

“…Similarly, equations (2) and (3) can be thought of as a variant of HLZ's “model with correlations” where all factors are assumed to be false, or a variant of Chen and Zimmermann (2020b) where all true returns are zero (see also Andrews and Kasy (2019)). Thus, while HLZ emphasize the concept of “multiple testing” and Chen and Zimmermann (2020b) emphasize “publication bias,” these concepts along with ‐hacking are simply the combination of conducting many tests (equation (2)) and reporting a selected subset (equation (3))).…”

“…These results come from the fact that signed predictive portfolios have correlations that are dispersed around zero (McLean and Pontiff (2016), Chen and Zimmermann (2020b)). These average‐zero correlations result in relatively small average effects on the conditional mean of t ‐statistics (equation (19)), but simultaneously imply that the conditional standard deviation is smaller than one (equation (20)), as the standard deviation adjustment involves squaring correlations.…”

“…I illustrate this generality by presenting several formal models that lead to equation (1), including a model of optimal ‐hacking with learning that follows Adam's (2001) extension of Weitzman's (1979) “Pandora's boxes” model. These models also show how equation (1) applies to concepts like data‐snooping (Lo and MacKinlay (1990)), multiple testing (Harvey, Liu, and Zhu (2016)), and publication bias (Chen and Zimmermann (2020b)).…”

“…In a study of 97 published cross‐sectional predictors, McLean and Pontiff (2016) find that predictability declines by only 26% in the first few years postsample. Chen and Zimmermann (2020b) find a decline of 23% for their 156 predictors, and Jacobs and Müller (2020) find a decline of 38% for a set of 241 predictors. While data far from the in‐sample period consistently produce larger declines, even these tests consistently find statistically significant predictability if all predictors are accounted for.…”