Primary healthcare (PHC) plays an important role in diabetes management; community health centers (CHCs) serve as the main providers of PHC. However, few studies have discussed the association between the service quality of PHC and the effects of diabetes management. In this study, we explored the associations between experiences of PHC in CHCs and glycemic control status in patients with diabetes mellitus. This study was conducted in six CHCs in the Greater Bay Area of China. In total, 418 patients with diabetes mellitus (44% males and 56% females) were recruited between August and October 2019. We evaluated their PHC experiences using the Primary Care Assessment Tool (PCAT) developed by Johns Hopkins and assessed their glycemic control status by measuring their fasting plasma glucose levels. Binary logistic regression analyses were conducted to assess the associations between the patients’ PHC experiences and glycemic control status, adjusting for covariates. The patients with good glycemic control had significantly higher total and dimensional PCAT scores compared with those with poor glycemic control (p < 0.05). Higher PCAT scores were significantly associated with a greater adjusted odds ratios (aORs) of good glycemic control for total and dimensional PCAT scores. For example, compared to those with poor glycemic control, the aORs for those with good glycemic control was 8.82 (95% CI = 4.38–17.76) per total PCAT score increasing. Especially, the aORs for those with good glycemic control were 3.92 (95% CI = 2.38–6.44) and 4.73 (95% CI = 2.73–8.20) per dimensional PCAT score of family-centeredness and community orientation increasing, respectively. Better PHC experiences were associated with better diabetes management. In particular, family-centered and community-oriented CHCs may help improve diabetes management in China and other low- and middle-income countries.