In the perspective of developing smart hybrid materials with customized features, ferrogels and magnetorheological elastomers allow a synergy of elasticity and magnetism. The interplay between elastic and magnetic properties gives rise to a unique reversible control of the material behavior by applying an external magnetic field. Albeit few works have been performed on the time-dependent properties so far, understanding the dynamic behavior is the key to model many practical situations, e.g., applications as vibration absorbers. Here we present a way to calculate the frequency-dependent elastic moduli based on the decomposition of the linear response to an external stress in normal modes. We use a minimal three-dimensional dipole-spring model to theoretically describe the magnetic and elastic interactions on the mesoscopic level. Specifically, the magnetic particles carry permanent magnetic dipole moments and are spatially arranged in a prescribed way, before they are linked by elastic springs. An external magnetic field aligns the magnetic moments. On the one hand, we study regular lattice-like particle arrangements to compare with previous results in the literature. On the other hand, we calculate the dynamic elastic moduli for irregular, more realistic particle distributions. Our approach measures the tunability of the linear dynamic response as a function of the particle arrangement, the system orientation with respect to the external magnetic field, as well as the magnitude of the magnetic interaction between the particles. The strength of the present approach is that it explicitly connects the relaxational modes of the system with the rheological properties as well as with the internal rearrangement of the particles in the sample, providing new insight into the dynamics of these remarkable materials.