Horizontal transfer of entire chromosomes has been reported in several fungal pathogens, often significantly impacting the fitness of the recipient fungus. All documented instances of horizontal chromosome transfers (HCTs) showed a marked propensity for accessory chromosomes, consistently involving the transfer of an accessory chromosome while other chromosomes were seldom, if ever, co-transferred. The mechanisms underlying HCTs, as well as the factors regulating the specificity of HCTs for accessory chromosomes, remain unclear. In this perspective, we provide an overview of the observed propensity in reported cases of horizontal chromosome transfers. We hypothesize the existence of a signal that distinguishes mobile, i.e., horizontally transferred, accessory chromosomes from the rest of the donor genome. Recent findings in Metarhizium robertsii and Magnaporthe oryzae, suggest that a mobile accessory chromosome may contain putative histones and/or histone modifiers, which could generate such a signal. Based on this, we propose that mobile accessory chromosomes may encode the machinery required for their own horizontal transmission, implying that HCT could be a regulated process. Finally, we present evidence of substantial differences in codon usage bias between core and accessory chromosomes in 14 out of 19 analysed fungal species and strains. Such differences in codon usage bias could indicate past horizontal transfers of these accessory chromosomes. Interestingly, HCT was previously unknown for many of these species, suggesting that the horizontal transfer of accessory chromosomes may be more widespread than previously thought and, therefore, an important factor in fungal genome evolution.