The attenuation of long-wavelength phonons (waves) by glassy disorder plays a central role in various glass anomalies, yet it is neither fully characterized, nor fully understood. Of particular importance is the scaling of the attenuation rate Γ(k) with small wavenumbers k → 0 in the thermodynamic limit of macroscopic glasses. Here we use a combination of theory and extensive computer simulations to show that the macroscopic low-frequency behavior emerges at intermediate frequencies in finite-size glasses, above a recently identified crossover wavenumber k † , where phonons are no longer quantized into bands. For k < k † , finite-size effects dominate Γ(k), which is quantitatively described by a theory of disordered phonon bands. For k > k † , we find that Γ(k) is affected by the number of quasilocalized nonphononic excitations, a generic signature of glasses that feature a universal density of states. In particular, we show that in a frequency range in which this number is small, Γ(k) follows a Rayleigh scattering scaling ∼ kd +1 (d is the spatial dimension), and that in a frequency range in which this number is sufficiently large, the recently observed generalized-Rayleigh scaling of the form ∼ kd +1 log(k0/k) emerges (k0 > k † is a characteristic wavenumber). Our results suggest that macroscopic glasses -and, in particular, glasses generated by conventional laboratory quenches that are known to strongly suppress quasilocalized nonphononic excitations -exhibit Rayleigh scaling at the lowest wavenumbers k and a crossover to generalized-Rayleigh scaling at higher k. Some supporting experimental evidence from recent literature is presented.