This paper draws support from the 2018 wave of the China Family Panel Studies (CFPS 2018) and uses unconditional quantile regression, re-centered influence function (RIF) decomposition, linear structural equation modelling, extended regression modelling and censored regression to explore the heterogeneity of the impact of Internet use on the psychological well-being of Chinese non-agricultural and agricultural hukou holders. We find that Internet use better improves the psychological well-being of non-agricultural hukou holders, thereby widening the gap in psychological well-being between urban and rural residents in China. Through RIF decomposition, we observe that, except for the 10th quantile, the expansion effect of Internet use on the inequality in psychological well-being between agricultural and non-agricultural hukou holders is mainly reflected in the structure effect, which shows that compared to non-agricultural hukou holders, the return rate of Internet use on the psychological well-being of agricultural hukou holders is lower. Further mechanism analysis shows that using the Internet to socialize, obtain information and understand politics is more beneficial for the psychological well-being of non-agricultural hukou holders; moreover, Internet use can further exert different effects on the psychological well-being of the two groups by differently influencing their job satisfaction, government evaluation, and sleep quality. This study also confirms that relying only on external scientific and technological progress has a limited corrective effect on existing inequalities.