We construct integrable generalizations of the elliptic Calogero-SutherlandMoser model of particles with spin, involving noncommutative spin interactions. The spin coupling potential is a modular function and, generically, breaks the global spin symmetry of the model down to a product of U (1) phase symmetries. Previously known models are recovered as special cases.