Given a holomorphic family f : X → S of compact complex manifolds and a relatively ample line bundle L → X , the higher direct images R n−p f * Ω p X /S (L) carry a natural hermitian metric. We give an explicit formula for the curvature tensor of these direct images. This generalizes a result of Schumacher [Sch12], where he computed the curvature of R n−p f * Ω p X /S (K ⊗m X /S ) for a family of canonically polarized manifolds. For p = n, it coincides with a formula of Berndtsson obtained in [Be11]. Thus, when L is globally ample, we reprove his result [Be09] on the Nakano positivity of f * (K X /F ⊗ L).