We present an analysis of asymmetries in global Hi spectra from the extended GALEX Arecibo SDSS Survey (xGASS), a stellar mass-selected and gas fraction-limited survey which is representative of the Hi properties of galaxies in the local Universe. We demonstrate that the asymmetry in a Hi spectrum is strongly linked to its signal-tonoise meaning that, contrary to what was done in previous works, asymmetry distributions for different samples cannot be compared at face value. We develop a method to account for noise-induced asymmetry and find that the typical galaxy detected by xGASS exhibits higher asymmetry than what can be attributed to noise alone, with 37% of the sample showing asymmetry greater than 10% at an 80% confidence level. We find that asymmetric galaxies contain, on average, 29% less Hi mass compared to their symmetric counterparts matched in both stellar mass and signal-to-noise. We also present clear evidence that satellite galaxies, as a population, exhibit more asymmetric Hi spectra than centrals and that group central galaxies show a slightly higher rate of Hi asymmetries compared to isolated centrals. All these results support a scenario in which environmental processes, in particular those responsible for gas removal, are the dominant driver of asymmetry in xGASS.