We develop a theory of p-adic continued fractions for a quaternion algebra B over Q ramified at a rational prime p. Many properties holding in the commutative case can be proven also in this setting. In particular, we focus our attention on the characterization of elements having a finite continued fraction expansion. By means of a suitable notion of quaternionic height, we prove a criterion for finiteness. Furthermore, we draw some consequences about the solutions of a family of quadratic polynomial equations with coefficients in B.