Skill upgrading, the increase in the percentage of skilled workers in the employment population, boosts the economic growth of developing countries and sustains their industrial competitiveness. The international economics literature discusses the effects of international trade on skill upgrading, ignoring the potential role of agglomeration externalities. This paper takes China as a case study, which has been encountering a serious challenge about how to strengthen its industrial competitiveness in the world through skill upgrading as its population dividend decreases. The panel data of 2005, 2010 and 2015 from prefecture-level cities in China were used for regression analysis to explore the benefits from agglomeration externalities, including specialization and diversification effects, on skill upgrading. The results show that both the specialization effect and diversification effect do promote skill upgrading. Furthermore, there are significant differences in the influence of local agglomeration externalities across different regions, and the positive effect brought about by specialization externalities is usually dominant in undeveloped, inland or small cities, compared with the diversification in developed or coastal cities. Besides, manufacturing agglomerations exhibit positive externalities to skill upgrading mainly through specialization, while the service agglomerations mainly promote skill upgrading by means of diversification.