College graduates, as a labor force with high human capital accumulation, have the problem of initial wage inequality, which is worth paying attention to. Based on the collated micro-survey data form “Employment and Entrepreneurship Development Report of Chinese College Graduates”, which contains 339 samples from vocational colleges, 453 from common colleges, and 360 from key colleges, this study empirically analyzed the inequality of college graduates’ initial wages at the college level. We found that the initial wage income level of college graduates is significantly influenced by the college level. The higher the level is, the higher the initial wage. The initial wage of graduates from key colleges is the highest, and the income inequality between them and vocational college graduates is the most significant. Moreover, there are structural differences in the wage premium effect of the college level on college graduates with a change in wage level. In addition, the study found that there is an obvious gender wage difference among college graduates, and political status, academic ranking, and student cadre experience as well as the nature of the workplace all contributed to the formation of wage premiums to a certain extent.