We present and compare several cosmological constraints on the cross section for elastic scattering between dark matter (DM) and baryons, for cross sections with a range of power-law dependences on the DM-baryon relative velocity v, especially focusing on the case of σ ∝ v −4 . We study constraints spanning a wide range of epochs in cosmological history, from prerecombination distortions to the blackbody spectrum and anisotropies of the cosmic microwave background (CMB), to modifications to the intergalactic medium temperature and the resulting 21 cm signal, and discuss the allowed signals in the latter channels given the constraints from the former. We improve previous constraints on DM-baryon scattering from the CMB anisotropies, demonstrate via principal component analysis that the effect on the CMB can be written as a simple function of DM mass for v −4 scattering, and map out the redshifts dominating this signal. We show that given high-redshift constraints on DM-baryon scattering, a v −4 scaling of the cross section for light DM would be sufficient to explain the deep 21 cm absorption trough recently claimed by the Experiment to Detect the Reionization Step, if 100% of the DM scatters with baryons; we estimate the minimal necessary velocity scaling and find that it is close to v −4 . For millicharged DM models proposed to explain the observation, where only a small fraction of the DM interacts, we estimate that a PIXIE-like future experiment measuring CMB spectral distortion could test the relevant parameter space.