We consider Modified Gravity models involving inverse powers of fourth-order curvature invariants.Using these models' equivalence to the theory of a scalar field coupled to a linear combination of the invariants, we investigate the properties of the propagating modes. Even in the case for which the fourth derivative terms in the field equations vanish, we find that the second derivative terms can give rise to ghosts, instabilities, and superluminal propagation speeds. We establish the conditions which the theories must satisfy in order to avoid these problems in Friedmann backgrounds, and show that the late-time attractor solutions generically exhibit superluminally propagating tensor or scalar modes. * a. de-felice@sussex.ac.uk