Sex and sexual differentiation are ubiquitous across the tree of life. Because females and males often have substantially different functional requirements, we would expect selection to differ between the sexes. Recent studies in diverse species, including humans, suggest sexually antagonistic viability selection creates allele frequency differences between the sexes at many different loci. However, theory and population-level simulations suggest that sex-specific differences in viability would need to be very extreme in order to produce and maintain reported levels of between-sex genetic differentiation. In this study, we evaluate evidence for human sexually antagonistic selection on genomic variation in two independent, large biobanks (BioVU, n = 93,864, and UK Biobank, n = 438,427). We performed association tests between genetically ascertained sex and genotypes and, while we found dozens of genome-wide significant associations, none replicated across samples. Moreover, closer inspection revealed that all associations are likely due to cross-hybridization with regions of the sex chromosome during genotyping. Therefore, we find no compelling evidence for sexual antagonism on the autosomes, despite being well-powered to detect sex-specific allelic differences of down to 0.8% between the sexes. This study not only demonstrates a lack of strong ongoing sexually antagonistic selection on variation in humans, but also highlights significant challenges in identifying the genetic basis of sex-specific fitness differences using genome-wide scans in any species.