We consider a classical fermion and a classical scalar, propagating on two different kinds of 4-dimensional diffeomorphism breaking gravity backgrounds, and we derive the one-loop effective dispersion relation for matter, after integrating out gravitons. One gravity model involves quadratic divergences at one-loop, as in Einstein gravity, and the other model is the z = 3 non-projectable Horava-Lifshitz gravity, which involves logarithmic divergences only. Although these two models behave differently in the UV, the IR phenomenology for matter fields is comparable: (i) for generic values for the parameters, both models identify 10 10 GeV as the characteristic scale above which they are not consistent with current upper bounds on Lorentz symmetry violation; (ii) for both models, there is always a fine-tuning of parameters which allows the cancellation of the indicator for Lorentz symmetry violation.