China’s rapid development and urbanization have created large numbers of migrant laborers, with increasing numbers of young adults and couples migrating from rural areas to large cities. As a result, a large number of children have become left-behind children (LBC), who were left behind in their hometown and cared for by one parent, grandparents, relatives or friends. Some of these LBC have a chance to be college students, who are called college students with left-behind experience. Some studies have indicated that the absence of these college students’ parents during childhood may cause them to have some mental health problems. Therefore, we want to examine the effects of left-behind experience on college students’ mental health and compare the prevalence of mental health problems in left-behind students and control students (without left-behind experience). For this purpose, a cross-sectional comparative survey was conducted in a coastal city of Shandong province, Eastern China. First, 1605 college students from three universities (national admissions) were recruited, including 312 students with left-behind experience and 1293 controls. Their mental health level was measured using Symptom Check-list 90 (containing ten dimensions: somatization, obsessive-compulsion (OCD), interpersonal sensitivity, depression, anxiety, hostility, terror, paranoia, psychoticism, and other symptoms). The results showed that left-behind experience was a significant risk factor for the mental health problems of college students (OR = 2.27, 95%CI: 1.73 to 2.97). A comparison of the two groups, after controlling the confounding factors using the coarsened exact matching (CEM) algorithm, showed that the prevalence of mental health problems was 35.69% (n = 311) among the left-behind students, while it was 19.68% (n = 1194) among the controls. The two groups were significantly different in terms of these ten dimensions of the SCL-90 scale (p < 0.001), and the prevalence of each dimension among the left-behind students was consistently higher than that among the controls. In addition, different left-behind experiences and social supports during childhood had different effects on mental health problems.