Background: Many studies focused on health inequality between migrant older adults and local older adults, while few study concerned the health inequalities between urban-to-urban and rural-to-urban migrant older adults. This study aimed to compare physical health and mental health between these two groups in Hangzhou, Zhejiang Province, China, and to explore the relationship between cognitive social capital, social integration and health among migrant older adults. Methods: A two-stage stratified sampling method was employed to recruit participants from May to August 2013 in Hangzhou. Measurement data were compared with student's t-tests and multivariate analysis of variance (MANOVA). Multiple linear regression was adopted in this study. Results: A total of 1000 of participants who met the inclusion criteria were analyzed, consisting of 527 (52.7%) urban-to-urban and 473 (47.3%) rural-to-urban migrant older adults. There were no statistically significant difference in physical health and mental health between urban-to-urban and rural-to-urban groups on the whole. However, urban-to-urban migrant older adults had a higher reciprocity and social integration than did in rural-to-urban group (13.36 vs. 12.50, p < 0.01; 40.07 vs. 38.50, p < 0.01). And both of cognitive social capital and social integration were positively related to physical health (social reciprocity: t = 6.69, p < 0.01; social trust: t = 3.27, p < 0.01; social integration: t = 5.66, p < 0.01) and mental health (social reciprocity: t = 4.49, p < 0.01; social trust: t = 5.15, p < 0.01; social integration: t = 10.02, p < 0.01). Overall, the female, widowed, and the oldest among migrant older adults had a worse health. Conclusions: Social capital and social integration were played important roles in health of migrant older adults. The female rural-to-urban migrant older adults, those aged over 70 years, and older adults who were not in marriage should be especially concerned in health policy making.