China’s rural older are the threat from chronic diseases, making it important to evaluate the effect of public health insurance on the health care utilization and expenditures with chronic diseases. In 2003, China initiated a public health insurance, which was called the New Rural Cooperative Medical System (NRCMS). NRCMS is a voluntary program, targeting rural residents with government subsidies and individual contribution. Using the two-stage residual inclusion approach (2SRI), we analyzed the impact of NRCMS on health-care service utilization and expenditure of rural older with chronic diseases by using the 2011 and 2013 China Health and Retirement Survey (CHARLS) data. The results showed NRCMS did not play an effective role on improving the medical services utilization of rural older with chronic diseases. Although NRCMS immediate reimbursement significantly reduced the outpatient service fee, the actual outpatient reimbursement is the opposite. In addition, NRCMS did not significantly decrease their hospitalization expense. Policy makers should pay attention to health management about chronic diseases in rural China, and some measures should be taken to deepen the medical security system reform and improve the public health service system.