BackgroundIn recent decades, China has experienced tremendous economic growth and also witnessed growing socioeconomic-related health inequality. The study aims to explore the potential causes of socioeconomic-related health inequality in urban and rural areas of China over the past two decades.MethodsThis study used six waves of the China Health and Nutrition Survey (CHNS) from 1991 to 2006. The recentered influence function (RIF) regression decomposition method was employed to decompose socioeconomic-related health inequality in China. Health status was derived from self-rated health (SRH) scores. The analyses were conducted on urban and rural samples separately.ResultsWe found that the average level of health status declined from 1989 to 2006 for both urban and rural populations. Average health scores were greater for the rural population compared with those for the urban population. We also found that there exists pro-rich health inequality in China. While income and secondary education were the main factors to reduce health inequality, older people, unhealthy lifestyles and a poor home environment increased inequality. Health insurance had the opposite effects on health inequality for urban and rural populations, resulting in lower inequality for urban populations and higher inequality for their rural counterparts.ConclusionThese findings suggest that an effective way to reduce socioeconomic-related health inequality is not only to increase income and improve access to health care services, but also to focus on improvements in the lifestyles and the home environment. Specifically, for rural populations, it is particularly important to improve the design of health insurance and implement a more comprehensive insurance package that can effectively target the rural poor. Moreover, it is necessary to comprehensively promote the flush toilets and tap water in rural areas. For urban populations, in addition to promoting universal secondary education, healthy lifestyles should be promoted, including measures such as alcohol control.Electronic supplementary materialThe online version of this article (doi:10.1186/s12939-017-0624-9) contains supplementary material, which is available to authorized users.