Introduction Childhood diarrhea remains a leading cause of death in children under 5 years of age worldwide, although the disease is both preventable and treatable. Effective caregiver involvement, rotavirus vaccinations, and parallel diarrhea prevention are critical. Objectives This study aimed to assess caregiver's knowledge of childhood diarrheal disease prevention, rotavirus vaccination knowledge, and associated factors in Dodoma, Tanzania. Methods A hospital-based analytical cross-sectional study was conducted at Makole Health Center, Dodoma. Data were collected from 274 caregivers of children under 5 years of age who attended a routine immunization clinic. Respondents were interviewed using a semi-structured, pretested questionnaire. Data were analyzed for frequency, percentages, and mean, followed by multivariable linear logistic regression to examine the association between sociodemographic factors and caregiver's knowledge. Results A total of 274 subjects were involved in the study. The majority (93.8%) were women, all had formal education, (62.8%) knew that diarrhea in children can be prevented by proper handwashing, and only (9.9%) knew about the rotavirus vaccine. Educational level (β = 0.565 CI; 0.490, 0.873) p-value < .001, age of parents (β = 0.738 CI; 0.427, 1.050) p-value .022 and age of child (β = −0.306 CI; − 0.490, −0.122), p-value <.001 and the fact that the child was male (β = 1.116, 95% CI = 0.712, 1.520) p-value < .001) were the factors associated with participants’ knowledge. Conclusion The results highlight the great need for public health programs to increase awareness of rotavirus vaccines and parallel diarrhea prevention interventions, which in turn will significantly improve caregivers’ knowledge and therefore the prevention of diarrhea in children