Yaks live in the harsh environment of the Qinghai–Tibet Plateau, and the cold climate causes lower growth efficiency. The aim of this experiment was to explore the effects of drinking warm water on the growth performance in yak calves and investigate the underlying physiological mechanisms. A total of 24 Datong yak calves were selected and randomly assigned into the cold water group (group C, water temperature around 0–10 °C without any heating; 58.03 ± 3.111 kg) and the warm water group (group W, water constantly heated at 2 °C; 59.62 ± 2.771 kg). After the 60-day experiment, body weight was measured, and rumen fluid and blood serum samples were collected for analysis. The results show that the body weight and average daily gain of yaks that drank warm water were higher compared to those that drank cold water (p < 0.05). The acetic, propionic, isobutyric, valeric, and isovaleric acid concentrations were higher in group W than in group C (p < 0.05). Additionally, warm water changed the ruminal microbes at different levels. At the phylum level, the relative abundance of Tenericutes, Kiritimatiellaeota, and Elusimicrobiota was higher in group C (p < 0.05). At the genus level, three genera were increased by warm water, including Ruminococcoides and Eubacteriales Family XIII. Incertae Sedis, and 12 genera were decreased, including Ruminococcus (p < 0.05). At the species level, unclassified Prevotellaceae and Ruminococcoides bili were increased by warm water compared to cold water (p < 0.05). According to the metabolomics results, metabolites, including valine, isoleucine, PC (15:0/22:2(13Z,16Z)), and LysoPC (18:0/0:0), were increased in the warm water group compared to the cold water group (p < 0.05), and were enriched in glycerophospholipid and amino acid metabolism pathways. This study analyzed the differences in ruminal microbes and metabolomes of yak calves provided with water at different temperatures and revealed the potential mechanism for better performance promoted by warm drinking water.