Stein's method is a collection of tools for analysing distributional comparisons through the study of a class of linear operators called Stein operators. Originally studied in probability, Stein's method has also enabled some important developments in statistics. This early success has led to a high research activity in this area in recent years. The goal of this survey is to bring together some of these developments in theoretical statistics as well as in computational statistics and, in doing so, to stimulate further research into the successful field of Stein's method and statistics. The topics we discuss include: explicit error bounds for asymptotic approximations of estimators and test statistics, a measure of prior sensitivity in Bayesian statistics, tools to benchmark and compare sampling methods such as approximate Markov chain Monte Carlo, deterministic alternatives to sampling methods, control variate techniques, and goodness-of-fit testing.