We study a quantum model of dynamical Casimir effect in an optical cavity enclosed by a freely moving mirror attached to a harmonic spring. The quantum fluctuations of the friction force exerted by the dynamical Casimir emission onto the moving mirror are investigated, as well as their consequences on the quantum state of the mirror oscillation. Observable signatures of the interplay of the nonlinear nature of the effective mirror-cavity coupling and of the discreteness of the emitted photons are pointed out, in particular as a fast diffusion of the mirror oscillation phase. These results are interpreted in the language of quantum field theories on curved space-times as a breakdown of the standard semiclassical theory of the backreaction.