Unemployment has serious negative effects on psychological health, and yet the underlying mechanisms remain unknown. According to the latent deprivation model, it is the lack of latent benefits of work associated with unemployment, which leads to psychological distress. In a four-wave study among employed persons, unemployed persons, and persons out of the labour force (OLF) (N T1 = 1,026), this assumption was tested cross-sectionally as well as longitudinally. Multiple mediation analyses show the expected differences in distress between the employed, unemployed, and OLF persons and indicate that part of this difference can be explained by differential access to the latent benefits. Furthermore, cross-lagged structural equation modelling confirms that a deprivation of latent benefits leads to a decrease in psychological health 6 months later. Findings regarding the different quality of the latent benefits in relation to each other and over time when predicting psychological health are discussed.