At present, the dual pressure of rural labor outflow and population aging in China makes the problems of the rural elderly population increasingly prominent, and its health problem is particularly prominent. Based on the 2014 China elderly population health survey data (CLHLS), this paper finds that the physical health status of the rural elderly has a significant positive impact on their loneliness; that is, the rural elderly with poor health status are more likely to feel lonely. At the same time, the age of the elderly has a significant positive impact on their loneliness. On the contrary, gender, personality, family income and intergenerational support of the elderly have a negative impact on their loneliness. Chronic diseases such as hypertension and diabetes have no significant effect on the loneliness of the elderly in rural areas, but there is a “severe disease effect”; that is, when chronic diseases develop into serious diseases or acute serious diseases, it can negatively impact the elderly psychologically and produce or deepen their sense of loneliness. Based on the above conclusions, this paper further puts forward relevant policy suggestions from three aspects: constructing a disease prevention and control system for the rural elderly, improving the care and service system for the rural elderly, reshaping rural filial piety culture, and creating a good atmosphere of “respecting, loving and respecting parents” in rural areas.