Predicting What is Predictable

Abstract

Coincidental correlations are useless for making predictions. In order to predict something, it has to be predictable; there must be an underlying causal structure—a real reason for the correlation. Correlation without causation mean predictions without hope. Causation can be demonstrated by a randomized controlled trial (RCT) in which there is both a treatment group and a control group, and in which the subjects are randomly assigned to the two groups. There should also be enough data to draw meaningful conclusions. A/B tests are essentially RCTs for the Internet. Unfortunately, we often cannot do RCTs. We have to make do with observational data. A valid study specifies the theory to be tested before looking at the data. Finding a pattern after looking at the data is treacherous, and likely to end badly—with a worthless, temporary coincidental correlation.