Beef cattle phenotypes are affected by the consumption of toxic fescue. Toxic fescue’s impact is dependent on heat stress and breed composition, with genetic variability for robustness to toxin exposure believed to exist within and across breeds. The study objective was to characterize the effect of fescue toxicosis across breeds for known and novel heat and fescue stress-associated phenotypes. One-hundred crossbred fall-calving Charolais- and Hereford-sired cows of parities 1–3 were allocated to graze either toxic fescue (n = 50), non-toxic fescue (n = 25), or a rotation between toxic and non-toxic fescue (n = 25) for 156 days. Phenotypes impacted by breed (genetics) included hair coat score (p < 0.0001), hair reduction/shedding rate (p < 0.05), rectal temperature (RT) (p < 0.0001), vaginal temperature (p < 0.05), serum phosphorus concentration (p < 0.02) and respiration rate (RR) (p < 0.003). Cows on toxic fescue experienced reduced hair shedding efficacy (p < 0.0001), higher vaginal temperatures (p < 0.0001), increased systolic blood pressure (p < 0.04), increased RR (p < 0.0001) and reduced average daily gain (p < 0.0001), compared to cows grazing non-toxic fescue. Calves born from cows with higher RT during the last third of gestation had higher RT at weaning (p < 0.02), indicating potential physiological effects of in utero heat stress. The study indicates that beef cows exhibit variable responses to toxic fescue within and across breeds which may impact future calf phenotypes.