Targeting the gastrointestinal microbiome for improvement of feed efficiency and reduction of production costs is a potential promising strategy. However little progress has been made in manipulation of the gut microbiomes in dairy cattle to improve milk yield and milk quality. Even less understood is the milk microbiome. Understanding the milk microbiome may provide insight into how the microbiota correlate with milk yield and milk quality. The objective of this study was to characterize similarities between rumen, fecal, and milk microbiota simultaneously, and to investigate associations between microbiota, milk somatic cell count (SCC), and milk yield. A total of 51 mid-lactation, multiparous Holstein dairy cattle were chosen for sampling of ruminal, fecal, and milk contents that were processed for microbial DNA extraction and sequencing. Cows were categorized based on low, medium, and high SCC; as well as low, medium, and high milk yield. Beta diversity indicated that ruminal, fecal, and milk populations were distinct (p < 0.001). Additionally, the Shannon index demonstrated that ruminal microbial populations were more diverse (p < 0.05) than were fecal and milk populations, and milk microbiota was the least diverse of all sample types (p < 0.001). While diversity indices were not linked (p > 0.1) with milk yield, milk microbial populations from cows with low SCC demonstrated a more evenly distributed microbiome in comparison to cows with high SCC values (p = 0.053). These data demonstrate the complexity of host microbiomes both in the gut and mammary gland. Further, we conclude that there is a significant relationship between mammary health (i.e., SCC) and the milk microbiome. Whether this microbiome could be utilized in efforts to protect the mammary gland remains unclear, but should be explored in future studies.