Problematic smartphone use (PSU) is a novel manifestation of addictive behaviors. It is frequently reported to be correlated with anxiety symptoms among University students. However, the underlying mechanism has not yet been thoroughly studied. Whether the association between anxiety symptoms and PSU is mediated or moderated by self-efficacy remains unclarified. A cluster sampling cross-sectional study was thus conducted to explore the potential mediating or moderating effect of self-efficacy in Chinese University students. Participants (N = 1,113) were recruited from eight Universities in Shenyang, China. Of them, 146 did not effectively respond to the questionnaires. Thus, 967 participants were eligible for the final analysis. The mediating or moderating role of self-efficacy in the anxiety-PSU relationship was explored using hierarchical multiple regression. Then the mediation model was further verified using the SPSS macros program (PROCESS v3.0). Our results showed that anxiety symptoms was positively correlated with PSU (r = 0.302, P < 0.01), while self-efficacy was negatively correlated with anxiety symptoms and PSU (r = −0.271 and −0.181, P < 0.01). Self-efficacy partly mediated the relationship between anxiety symptoms and PSU, which accounted for ~17.5% of the total effect that anxiety symptoms have on PSU. However, the moderating effect of self-efficacy on the anxiety-PSU relationship was insignificant. In summary, our findings suggested that self-efficacy partly mediates but not moderates the link between anxiety symptoms and PSU among Chinese University students. Therefore, multicomponent interventions should be made to restrict the frequency of smartphone usage, enhance the level of self-efficacy, and thus promote the mental health status of University students.