Purpose This study aimed to explore sex differences in the association between emotional support and self-rated health among the elderly. Design This was a cross-sectional survey based on the sub-project of China’s National Basic Public Health Service Project—Health Management Services for the Elderly. Setting Participants were recruited from ten rural townships in Jingyuan County, Gansu Province, Northwestern China. Subjects 1405 subjects aged 60 or above. Methods Emotional support (consisting of 5 items) and self-rated health (evaluated by EQ-VAS) were investigated in this study. Multiple linear regression was conducted to consider the potential relationship. Results The frequency of children visit and the number of providers of emotional support were positively associated with self-rated health among older women (β = 1.13, 95%CI = 0.25-2.02; β = 1.80, 95%CI = 1.01-2.58), whereas the number of close friends had a positive association with self-rated health among older men (β = 1.11, 95%CI = 0.20-2.01). The number of close relatives and the frequency of seeking emotional support were not found to be associated with self-rated health among both older men and older women. Conclusion The study has found that the relationship between emotional support and self-rated health was differed by sex, calling attention to the need for sex-specific interventions.