We investigate theoretically the macroscopic dynamics of various types of ordered magnetic fluid, gel, and elastomeric phases. We take a symmetry point of view and emphasize its importance for a macroscopic description. The interactions and couplings among the relevant variables are based on their individual symmetry behavior, irrespective of the detailed nature of the microscopic interactions involved. Concerning the variables we discriminate between conserved variables related to a local conservation law, symmetry variables describing a (spontaneously) broken continuous symmetry (e.g., due to a preferred direction) and slowly relaxing ones that arise from special conditions of the system are considered. Among the relevant symmetries, we consider the behavior under spatial rotations (e.g., discriminating scalars, vectors or tensors), under spatial inversion (discriminating e.g., polar and axial vectors), and under time reversal symmetry (discriminating e.g., velocities from polarizations, or electric fields from magnetic ones). Those symmetries are crucial not only to find the possible cross-couplings correctly but also to get a description of the macroscopic dynamics that is compatible with thermodynamics. In particular, time reversal symmetry is decisive to get the second law of thermodynamics right. We discuss (conventional quadrupolar) nematic order, polar order, active polar order, as well as ferromagnetic order and tetrahedral (octupolar) order. In a second step, we show some of the consequences of the symmetry properties for the various systems that we have worked on within the SPP1681, including magnetic nematic (and cholesteric) elastomers, ferromagnetic nematics (also with tetrahedral order), ferromagnetic elastomers with tetrahedral order, gels and elastomers with polar or active polar order, and finally magnetorheological fluids and gels in a one- and two-fluid description.