Since inflationary perturbations must generically couple to all degrees of freedom present in the early Universe, it is more realistic to view these fluctuations as an open quantum system interacting with an environment. Then, on very general grounds, their evolution can be modelled with a Lindblad equation. This modified evolution leads to quantum decoherence of the system, as well as to corrections to observables such as the power spectrum of curvature fluctuations. On one hand, current cosmological observations constrain the properties of possible environments and place upper bounds on the interaction strengths. On the other hand, imposing that decoherence completes by the end of inflation implies lower bounds on the interaction strengths. Therefore, the question arises of whether successful decoherence can occur without altering the power spectrum. In this paper, we systematically identify all scenarios in which this is possible. As an illustration, we discuss the case in which the environment consists of a heavy test scalar field. We show that this realises the very peculiar configuration where the correction to the power spectrum is quasi scale invariant. In that case, the presence of the environment improves the fit to the data for some inflationary models but deteriorates it for others. This clearly demonstrates that decoherence is not only of theoretical importance but can also be crucial for astrophysical observations.