We examine a model of two interacting populations of phase oscillators labelled 'Blue' and 'Red'. To this we apply tempered stable Lévy noise, a generalisation of Gaussian noise where the heaviness of the tails parametrised by a power law exponent α can be controlled by a tempering parameter λ . This system models competitive dynamics, where each population seeks both internal phase synchronisation and a phase advantage with respect to the other population, subject to exogenous stochastic shocks. We study the system from an analytic and numerical point of view to understand how the phase lag values and the shape of the noise distribution can lead to steady or noisy behaviour. Comparing the analytic and numerical studies shows that the bulk behaviour of the system can be effectively described by dynamics in the presence of tilted ratchet potentials. Generally, changes in α away from the Gaussian noise limit, 1 < α < 2, disrupts the locking between Blue and Red, while increasing λ acts to restore it. However we observe that with further decreases of α to small values, α 1, with λ = 0, locking between Blue and Red may be restored. This is seen analytically in a restoration of metastability through the ratchet mechanism, and numerically in transitions between periodic and noisy regions in a fitness landscape using a measure of noise. This non-monotonic transition back to an ordered regime is surprising for a linear variation of a parameter such as the power law exponent and provides a novel mechanism for guiding the collective behaviour of such a complex competitive dynamical system.