Gravitational waves from a first-order cosmological phase transition, at temperatures at the MeV-scale, would arguably be the most exciting explanation of the common red spectrum reported by the NANOGrav collaboration, not the least because this would be direct evidence of physics beyond the standard model. Here we perform a detailed analysis of whether such an interpretation is consistent with constraints on the released energy deriving from big bang nucleosynthesis and the cosmic microwave background. We find that a phase transition in a completely secluded dark sector with sub-horizon sized bubbles is strongly disfavoured with respect to the more conventional astrophysical explanation of the putative gravitational wave signal in terms of supermassive black hole binaries. On the other hand, a phase transition in a dark sector that subsequently decays, before the time of neutrino decoupling, remains an intriguing possibility to explain the data. From the model-building perspective, such an option is easily satisfied for couplings with the visible sector that are small enough to evade current collider and astrophysical constraints. The first indication that could eventually corroborate such an interpretation, once the observed common red spectrum is confirmed as a nHz gravitational wave background, could be the spectral tilt of the signal. In fact, the current data already show a very slight preference for a spectrum that is softer than what is expected from the leading astrophysical explanation.