Aspergillus fumigatus is both an environmental saprobe and an opportunistic human fungal pathogen. Knowledge of genomic variation across A. fumigatus isolates is essential for understanding the evolution of pathogenicity, virulence, and resistance to antifungal drugs. Here, we investigated 206 A. fumigatus isolates (133 clinical and 73 environmental isolates) aiming to identify genes with variable presence across isolates and test whether this variation was related to the clinical or environmental origin of isolates. The PanCore genome of A. fumigatus constitutes 13,085 ortholog groups, of which 7,773 (59.4%) are shared by all isolates (CORE) and 5,312 (40.6%) vary in their gene presence across isolates (ACCESSORY). Despite differences in the distribution of orthologs across all isolates, no significant differences were observed among clinical vs. environmental isolates when accounting for phylogeny. Orthologs that differ in their distribution across isolates tend to occur in low frequency and/or be restricted to specific isolates; thus, the degree of genomic conservation between orthologs of A. fumigatus is high. These results suggest that differences in the distribution of orthologs within A. fumigatus cannot be associated with the clinical or environmental origin of isolates.