Smartphone usage has brought great convenience to our daily life. However, the detrimental consequences of smartphone dependence also require special attention. Based on the theory of planned behavior (TPB), this study designed a questionnaire to investigate college students’ withdrawal from smartphone dependence. The reliability and validity of the questionnaire were tested. The results showed that the reliability of the questionnaire was good (α = 0.84–0.94). The principal component analysis was conducted, contribution rates of these factors were moderately high (62.06%–82.35%). The validity was verified through confirmatory factor analysis. All the model fit indices were higher than the acceptable standard, which indicated the construct validity of the TPB questionnaire is good. The results of the multi-group structural equation model revealed that the model provided excellent fit to the data in each demographic group. The results of structural equation model presented that, in most demographic groups, students’ attitude, subjective norm, and perceived behavioral control of withdrawal from smartphone dependence positively influenced their behavioral intention. Furthermore, subjective norm was the most effective predictors in each group. However, part of the paths from intention to behavior failed to reach the significant level, and their coefficients are relatively small. This reveals the gap between college students’ withdrawal intention and their actual behavior, and there may be other factors influence the relationship between the two variables.