In the US, rural communities face challenges to meet the community health needs of older adults and children. Meanwhile, rural areas lag in age-friendly built environment and services. AARP, a US based organization promoting livability for all ages, has developed a Livability Index based on the World Health Organization’s (WHO) domains of age-friendly communities: health, housing, neighborhood, transportation, environment, engagement, and opportunity. This study links the 2018 AARP Livability Index categories with demographic structure and socio-economic factors from the American Community Survey at the county level in the US to examine if the physical, built and social environment differentiate communities with better community health across the rural–urban divide. Results show that the neighborhood built environment has the largest impact on community health for all county types. Although rural areas lag in community health, those which give more attention to engagement and opportunity rank higher. Rural communities with more African Americans, children, and poor Whites, rank lower on community health. While neighborhood characteristics have the strongest link to community health, a broader approach with attention to age, race, poverty and engagement and opportunity is needed for rural areas.