In dioecious populations, the sexes employ divergent reproductive strategies to maximize fitness and, as a result, genetic variants can affect fitness differently in males and females. Moreover, recent studies have highlighted an important role of the mating environment in shaping the strength and direction of sex-specific selection. Here, we measure the sex-specific outbred fitness of 357 lines from the Drosophila Synthetic Population Resource (DSPR) in two different mating environments. We analyze the data using three different approaches to gain insight into the sex-specific genetic architecture for fitness: classical quantitative genetics, genomic associations, and a mutational burden approach. The quantitative genetics analysis finds that, on average segregating genetic variation in this population has concordant fitness effects both across the sexes and across mating environments. We do not find specific genomic regions with strong associations with either sexually antagonistic (SA) or sexually concordant (SC) fitness effects, yet there is modest evidence of an excess of genomic regions with weak associations, both with SA and SC fitness effects. Our examination of mutational burden indicates stronger selection against indels and loss-of-function variants in females than males.