The notion of pre-Leibniz algebras was recently introduced in the study of Rota-Baxter operators on Leibniz algebras. In this paper, we first construct a graded Lie algebra whose Maurer-Cartan elements are pre-Leibniz algebras. Using this characterization, we define the cohomology of a pre-Leibniz algebra with coefficients in a representation. This cohomology is shown to split the Loday-Pirashvili cohomology of Leibniz algebras. As applications of our cohomology, we study formal and finite order deformations of a pre-Leibniz algebra. Finally, we define homotopy pre-Leibniz algebras and classify some special types of homotopy pre-Leibniz algebras.