Nutrition transition in China has a strong impact on dietary quality and health of Chinese consumers. This study developed the diet quality divergence Index (DQD), the divergence between real food consumption and the Chinese food pagoda 2016 (CFP), to measure the quality of diet in China. Using four waves of data (2004, 2006, 2009, and 2011) from China Health and Nutrition Survey (CHNS), this study shed light on the transition of diet quality for Chinese residents. Results indicate that the DQD generally decreased and Chinese diet quality improved during 2004–2011. The divergence was mainly caused by over-consumption of legumes and nuts, and under-consumption of milk and milk products. Rising income and urbanization were positively correlated with diet quality for the people with low DQD. However, both of them had negative impacts on diet quality for those with high DQD. Females and rural residents held a lower DQD than their counterparts. The results also revealed that healthy food preference, education, dining at home, household size, proportions of teens (6–17) and elders (over 64) in the families are positively correlated with Chinese diet quality. However, labor intensity, frequency of drinking alcohol, and smoking have negative impacts on diet quality. Moreover, higher DQD was found to be associated with increasing risks of overweight/obesity. Therefore, we suggest national healthy policies should pay more attention to nutrition education. It is also necessary to focus on populations with poor diet quality and to adopt measures to control drinking alcohol and smoking.