Loose aggregations of fishes, or shoals, are a basal social organization of vertebrates and offer a valuable opportunity to determine how individual perceptions influence group formation. We used zebrafish, Danio rerio, to comprehensively investigate the preference space for shoaling related to adult pigment pattern variation, presented in the form of 17 zebrafish pigment pattern mutants or closely related species. We examined all combinations of these phenotypes in 2,920 initial and replicated preference tests, and used as subjects both domesticated laboratory stocks and wild-caught fish. By using multidimensional scaling and other approaches, we show that laboratory and wild zebrafish exhibit similar preferences, yet, unexpectedly, these preferences differ markedly between sexes, and also from how human observers perceive the same pigment patterns. Whereas zebrafish males respond to two traits (species and stripe patterning) in deciding whether to join a shoal, zebrafish female preferences do not correlate with a priori identifiable traits, and neither perceptual world is correlated with that of human observers. The observed zebrafish sex differences run counter to the most commonly accepted explanations for the individual selective advantages gained by shoaling. More generally, these data describe very different perceptual worlds between sexes and reveal the importance of sex differences in social group formation, as well as the critical importance of defining species specificity in visual signaling.perception ͉ pigment pattern ͉ shoaling ͉ social behavior ͉ zebrafish