We use a sample of star-forming field and protocluster galaxies at z=2.0-2.5 with Keck/MOSFIRE K-band spectra, a wealth of rest-frame ultraviolet (UV) photometry, and Spitzer/MIPS and Herschel/PACS observations, to dissect the relation between the ratio of infrared (IR) to UV luminosity (IRX) versus UV slope (β) as a function of gas-phase metallicity (+ 12 log O H ()∼8.2-8.7). We find no significant dependence of the IRX-β trend on environment. However, we find that at a given β, IRX is highly correlated with metallicity, and less correlated with mass, age, and specific star formation rate (sSFR). We conclude that, of the physical properties tested here, metallicity is the primary physical cause of the IRX-β scatter, and the IRX correlation with mass is presumably due to the mass dependence on metallicity. Our results indicate that the UV attenuation curve steepens with decreasing metallicity, and spans the full range of slope possibilities from a shallow Calzetti-type curve for galaxies with the highest metallicity in our sample (+ 12 log O H ()∼8.6) to a steep Small Magellanic Cloud (SMC)-like curve for those with + 12 log O H ()∼8.3. Using a Calzetti (SMC) curve for the low (high) metallicity galaxies can lead to up to a factor of 3 overestimation (underestimation) of the UV attenuation and obscured star formation rate. We speculate that this change is due to different properties of dust grains present in the interstellar medium of low-and high-metallicity galaxies. Unified Astronomy Thesaurus concepts: Interstellar dust extinction (837); Interstellar dust (836); Dust continuum emission (412); Galaxy evolution (594); Chemical abundances (224); Galaxy chemical evolution (580); Scaling relations (2031); Galaxy properties (615); Galaxy abundances (574); Star formation (1569)