The fruiting bodies, or mushrooms, of terrestrial fungi have been found to contain a high proportion of the nontoxic arsenic compound arsenobetaine (AB), but data gaps include a limited phylogenetic diversity of the fungi for which arsenic speciation is available, a focus on mushrooms with higher total arsenic concentrations, and the unknown formation and role of AB in mushrooms. To address these, the mushrooms of 46 different fungus species (73 samples) over a diverse range of phylogenetic groups were collected from Canadian grocery stores and background and arsenic-contaminated areas. Total arsenic was determined using ICP-MS, and arsenic speciation was determined using HPLC-ICP-MS and complementary X-ray absorption spectroscopy (XAS). The major arsenic compounds in mushrooms were found to be similar among phylogenetic groups, and AB was found to be the major compound in the Lycoperdaceae and Agaricaceae families but generally absent in log-growing mushrooms, suggesting the microbial community may influence arsenic speciation in mushrooms. The high proportion of AB in mushrooms with puffball or gilled morphologies may suggest that AB acts as an osmolyte in certain mushrooms to help maintain fruiting body structure. The presence of an As(III)-sulfur compound, for the first time in mushrooms, was identified in the XAS analysis. Except for Agaricus sp. (with predominantly AB), inorganic arsenic predominated in most of the store-bought mushrooms (albeit with low total arsenic concentrations). Should inorganic arsenic predominate in these mushrooms from contaminated areas, the risk to consumers under these circumstances should be considered.