We explore connections between covariance representations, Bismut-type formulas and Stein's method. First, using the theory of closed symmetric forms, we derive covariance representations for several well-known probability measures on R d , d ≥ 1. When strong gradient bounds are available, these covariance representations immediately lead to L p -L q covariance estimates, for all p ∈ (1, +∞) and q = p/(p − 1). Then, we revisit the well-known L p -Poincaré inequalities (p ≥ 2) for the standard Gaussian probability measure on R d based on a covariance representation. Moreover, for the nondegenerate symmetric α-stable case, α ∈ (1, 2), we obtain L p -Poincaré and pseudo-Poincaré inequalities, for p ∈ (1, α), via a detailed analysis of the various Bismut-type formulas at our disposal. Finally, using the construction of Stein's kernels by closed forms techniques, we obtain quantitative high-dimensional CLTs in 1-Wasserstein distance when the limiting Gaussian probability measure is anisotropic. The dependence on the parameters is completely explicit and the rates of convergence are sharp.