Gene gains/losses during evolution are critical for the adaptation of organisms to new environments or hosts. However, it remains unknown whether gene family expansions facilitated the adaptation of phytopathogenic fungi to woody plants. In this study, we compared the newly sequenced genome of the Colletotrichum gloeosporioides strain CFCC80308 with the genomes of two other C. gloeosporioides strains, Cg-14 and Lc-1, isolated from Persea americana and Liriodendron leaves, respectively. The genes in the expanded families, which were associated with plant surface signal recognition, encoded various proteins, including glycosyde hydrolases (GHs) and cytochrome P450. Interestingly, there was a substantial increase in the number of GH family genes in CFCC80308. Specifically, there were 368 enriched genes in the GH families (e.g., GH1, GH3, GH10, GH12, GH15, GH16, GH17, GH18, GH25, GH32, GH53, GH61, GH76, and GH81); the expression levels of these genes were highly up-regulated during the infection of poplar trees. Additionally, the GH17 family was larger in CFCC80308 than in C. gloeosporioides strains Cg-14 and Lc-1. Furthermore, the expansion of the MP65-encoding gene family during the adaptation of Colletotrichum species to woody plants was consistent with the importance of gene gains/losses for the adaptation of organisms to their environments. This study has clarified how C. gloeosporioides adapted to woody plants during evolution.