The rumen is a microbial-rich ecosystem in which rumen fungi play an important role in the feed digestion of ruminants. The composition of rumen fungi in free-range ruminants such as gayals, yaks, Tibetan yellow cattle, and the domesticated Yunnan yellow cattle was investigated by sequencing an internal transcribed spacer region 1 (ITS1) using Illumina MiSeq. A total of 285 092 optimized sequences and 904 operational taxonomic units (OTUs) were obtained from the four cattle breeds. The rumen fungi abundance and Chao and Simpson indexes were all higher in free-range ruminants than in domesticated ruminants. Three fungal phyla were identified by sequence comparison: Neocallimastigomycota, Basidiomycota, and Ascomycota. Basidiomycota and Ascomycota have very low abundance in the rumen of four breeds cattle but anaerobic fungi (AF) Neocallimastigomycota occurred in a high abundance. In Neocallimastigomycota, the dominant genera were Piromyces, Anaeromyces, Cyllamyces, Neocallimastix, and Orpionmyces in four cattle breeds. The composition of the major genera of Neocallimastigaceae varied greatly among the four cattle breeds. The unclassified genera were unequally distributed in gayals, yaks, Tibetan and Yunnan yellow cattle, accounting for 90.63%, 98.52%, 97.79%, and 27.01% respectively. It appears that free-range ruminants have more unknown rumen fungi than domesticated ruminants and the cattle breeds and animal diets had an impact on the diversity of rumen fungi.