BackgroundGenital diversity may arise through sexual conflict over polyandry, where male genital features function to manipulate female mating frequency against her interest. Correlated genital evolution across animal groups is consistent with this view, but a link between genital complexity and mating rates remains to be established. In sexually size dimorphic spiders, golden orbweaving spiders (Nephilidae) males mutilate their genitals to form genital plugs, but these plugs do not always prevent female polyandry. In a comparative framework, we test whether male and female genital complexity coevolve, and how these morphologies, as well as sexual cannibalism, relate to the evolution of mating systems.ResultsUsing a combination of comparative tests, we show that male genital complexity negatively correlates with female mating rates, and that levels of sexual cannibalism negatively correlate with male mating rates. We also confirm a positive correlation between male and female genital complexity. The macroevolutionary trajectory is consistent with a repeated evolution from polyandry to monandry coinciding with the evolution towards more complex male genitals.ConclusionsThese results are consistent with the predictions from sexual conflict theory, although sexual conflict may not be the only mechanism responsible for the evolution of genital complexity and mating systems. Nevertheless, our comparative evidence suggests that in golden orbweavers, male genital complexity limits female mating rates, and sexual cannibalism by females coincides with monogyny.Electronic supplementary materialThe online version of this article (doi:10.1186/s12862-016-0821-y) contains supplementary material, which is available to authorized users.