Recent observations from PAMELA, FERMI, and ATIC point to a new source of high energy cosmic rays. If these signals are due to annihilating dark matter, the annihilation cross section in the present day must be substantially larger than that necessary for thermal freeze-out in the early universe. A new force, mediated by a particle of mass O(100 MeV), leading to a velocity dependent annihilation cross section -a 'Sommerfeld enhancement' -has been proposed as a possible explanation. We point out that such models necessarily increase the dark matter (DM) self-scattering cross section, and use observational bounds on the amount of DM-DM scattering allowed in various astrophysical systems to place constraints on the mass and couplings of the light mediator.