Fertilization in angiosperms involves the germination of pollen on the stigma, followed by the extrusion of a pollen tube that elongates through the style and delivers two sperm cells to the embryo sac. Sexual selection could occur throughout this process when male gametophytes compete for fertilization. The strength of sexual selection during pollen competition should be affected by the number of genotypes deposited on the stigma. As increased self-fertilization reduces the number of mating partners, and the genetic diversity and heterozygosity of populations, it should thereby reduce the intensity of sexual selection during pollen competition. Despite the prevalence of mating system shifts, few studies have directly compared the molecular signatures of sexual selection during pollen competition in populations with different mating systems. Here we analyzed whole-genome sequences from natural populations of Arabis alpina, a species showing mating system variation across its distribution, to test whether shifts from cross- to self-fertilization result in molecular signatures consistent with sexual selection on genes involved in pollen competition. We found evidence for efficient purifying selection on genes expressed in vegetative pollen, and overall weaker selection on sperm-expressed genes. This pattern was robust when controlling for gene expression level and specificity. In agreement with the expectation that sexual selection intensifies under cross-fertilization, we found that the efficacy of purifying selection on male gametophyte-expressed genes was significantly stronger in genetically more diverse and outbred populations. Our results show that intra-sexual competition shapes the evolution of pollen-expressed genes, and that its strength fades with increasing self-fertilization rates.