The use of ultraviolet (UV) emission as a tracer of galaxy star-formation rate (SFR) is hampered by dust obscuration. The empirical relationship between UV slope, β, and the ratio between far-infrared and UV luminosity, IRX, is commonly employed to account for obscured UV emission. We present a simple model that explores the physical origin of variations in the IRX -β dust attenuation relation. A relative increase in FUV compared to NUV attenuation and an increasing stellar population age cause variations towards red UV slopes for a fixed IRX. Dust geometry effects (turbulence, dust screen with holes, mixing of stars within the dust screen, two-component dust model) cause variations towards blue UV slopes. Poor photometric sampling of the UV spectrum causes additional observational variations. We provide an analytic approximation for the IRX -β relation invoking a subset of the explored physical processes (dust type, stellar population age, turbulence). We discuss observed variations in the IRX -β relation for local (sub-galactic scales) and high-redshift (normal and dusty star-forming galaxies, galaxies during the epoch of reionization) galaxies in the context of the physical processes explored in our model. High spatial resolution imaging of the UV and sub-mm emission of galaxies can constrain the IRX -β dust attenuation relation for different galaxy types at different epochs, where different processes causing variations may dominate. These constraints will allow the use of the IRX -β relation to estimate intrinsic SFRs of galaxies, despite the lack of a universal relation.