Parents are the important implementers on appropriate/inappropriate use of antibiotics, especially in the pediatric population. Limited studies have associated poor knowledge, attitude, and practice (KAP) among parents with antibiotics misuse. Therefore, this study was conducted to determine the parents’ KAP and factors associated with inappropriate use of antibiotics among Tanzanian children. A hospital-based cross-sectional study was conducted in 14 regional referral hospitals (RRHs) in Tanzania between June and September 2020. KAP was estimated using a Likert scale, whereas KAP factors were determined using logistic regression models. A total of 2802 parents were enrolled in the study. The median age (interquartile range) of parents was 30.0 (25–36) years where 82.4% (n = 2305) were female parents. The majority of the parents had primary education, 56.1% (n = 1567). Of 2802 parents, only 10.9% (n = 298) had good knowledge about antibiotics, 16.4% (n = 455) had positive attitude whereas 82.0% (n = 2275) had poor practice on the appropriate use of antibiotics. Parents' education level, i.e., having a university degree (aOR: 3.27 95% CI 1.62–6.63, p = 0.001), good knowledge (aOR: 1.70, 95% CI 1.19–2.23, p = 0.003) and positive attitudes (aOR: 5.56, 95% CI 4.09–7.56, p < 0.001) were significantly associated with the appropriate use of antibiotics in children. Most parents had poor knowledge, negative attitude, and poor practice towards antibiotics use in children. Parents’ education level, employment status, knowledge on antibiotic use, and good attitude contributed to the appropriate use of antibiotics in children attending clinics at RRHs.