This paper concerns stochastic perturbations of piecewise-smooth ODE systems relevant for vibro-impacting dynamics, where impact events constitute the primary source of randomness. Such systems are characterized by the existence of switching manifolds that divide the phase space into regions where the system is smooth. The initiation of impacts is captured by a grazing bifurcation, at which a periodic orbit describing motion without impacts develops a tangential intersection with a switching manifold. Oscillatory dynamics near regular grazing bifurcations are described by piecewisesmooth maps involving a square-root singularity, known as Nordmark maps. We consider three scenarios where colored noise only affects impacting dynamics, and derive three two-dimensional stochastic Nordmark maps with the noise appearing in different nonlinear or multiplicative ways, depending on the source of the noise. Consequently the stochastic dynamics differs between the three noise sources, and is fundamentally different to that of a Nordmark map with additive noise. This critical dependence on the nature of the noise is illustrated with a prototypical one-degree-of-freedom impact oscillator.