We study the properties of the subgap states in p-wave superfluids, which occur at energies below the bulk gap and are localized inside the cores of vortices. We argue that their presence affects the topological protection of the zero modes. Transitions between the subgap states, including the zero modes and at energies much smaller than the gap, can alter the quantum states of the zero-modes. Consequently, qubits defined uniquely in terms of the zero-modes do not remain coherent, while compound qubits involving the zero-modes and the parity of the occupation number of the subgap states on each vortex are still well defined. In neutral superfluids, it may be difficult to measure the parity of the subgap states. We propose to avoid this difficulty by working in the regime of small chemical potential µ, near the transition to a strongly paired phase, where the number of subgap states is reduced. We develop the theory to describe this regime of strong pairing interactions and we show how the subgap states are ultimately absorbed into the bulk gap. Since the bulk gap also vanishes as µ → 0 there is an optimum value µc which maximises the combined gap. We propose cold atomic gases as candidate systems where the regime of strong interactions can be explored, and explicitly evaluate µc in a Feshbach resonant 40 K gas. In particular, the parameter c2 parametrizing the strength of the resonance in such gases, sets the characteristic size of vortices, and the energy scale of the subgap states.