Granular damping devices constitute an emerging technology for the attenuation of vibrations based on the dissipative nature of particle collisions. We show that the performance of such devices is independent of the material properties of the particles for working conditions where damping is optimal. Even the suppression of a dissipation mode (collisional or frictional) is unable to alter the response. We explain this phenomenon in terms of the inelastic collapse of granular materials. These findings provide a crucial standpoint for the design of such devices in order to achieve the desired low maintenance feature that makes particle dampers particularly suitable to harsh environments.