We give a new construction of p-adic heights on varieties over number fields using p-adic Arakelov theory. In analogy with Zhang's construction of real-valued heights in terms of adelic metrics, these heights are given in terms of p-adic adelic metrics on line bundles. In particular, we describe a construction of canonical p-adic heights an abelian varieties and we show that, for Jacobians, this recovers the height constructed by Coleman and Gross. Our main application is a new and simplified approach to the Quadratic Chabauty method for the computation of rational points on certain curves over the rationals, by pulling back the canonical height on the Jacobian with respect to a carefully chosen line bundle.