Many endurance athletes have poor sleep quality which may affect performance and health. It is unclear how dietary intake affects sleep quality among athletes. We examined if sleep quality in endurance athletes is associated with consumption of fruit, vegetables, whole grains, dairy milk, and caffeinated beverages. Two hundred thirty-four endurance athletes (39.5 ± 14.1 year) participated in a survey. Participants provided information on demographics, anthropometry, sleep behavior and quality, and dietary intake via questionnaires. Sleep quality was assessed using the Athlete Sleep Screening Questionnaire (ASSQ) with a global score (ASSQ-global) and subscales including sleep difficulty (ASSQ-SD), chronotype (ASSQ-C), and disordered breathing while sleeping (ASSQ-SDB). A general linear model (GLM), adjusted for age, body mass index, sleep discomfort, sleep behavior, gender, race, and ethnicity, showed that higher caffeinated beverage intake was related to poorer global sleep quality (p = 0.01) and increased risk for disordered breathing while sleeping (p = 0.03). Higher whole grain intake was associated with a morning chronotype and lower risk for sleep issues (p = 0.01). The GLM did not reveal a relationship between sleep quality and dairy milk, fruit, and vegetable intake. In conclusion, caffeinated beverages and whole grain intake may influence sleep quality. This relationship needs to be confirmed by further research.