In this work we study how CPT -odd Maxwell-Carroll-Field-Jackiw (MCFJ) electrodynamics as well as a dimension-5 extension of it affect the optical activity of continuous media. The starting point is dimension-3 MCFJ electrodynamics in matter whose modified Maxwell equations, permittivity tensor, and dispersion relations are recapitulated. Corresponding refractive indices are achieved in terms of the frequency and the vector-valued background field. For a purely timelike background, the refractive indices are real. Their associated propagation modes are circularly polarized and exhibit birefringence. For a purely spacelike background, one refractive index is always real and the other can be complex. The circularly polarized propagating modes may exhibit birefringence and dichroism (associated with absorption). Subsequently, we examine a dimension-five MCFJ-type electrodynamics, previously scrutinized in the literature, in a continuous medium. Following the same procedure, we find the refractive indices from a sixth-order dispersion equation. For a purely timelike background, three distinct refractive indices are obtained, one of them being real and two being complex. They are associated with two circularly polarized propagating modes that exhibit birefringence or dichroism, depending on the frequency range. Scenarios of propagation and absorption analogous to those found in dispersive dielectrics are also observed for spacelike background configurations. We conclude by comparing the dimension-three and five results and by emphasizing the richer phenomenology of the propagating modes in the higher-derivative model. Our results are applicable in the realm of Weyl semimetals.