BackgroundThe East African riverine cichlid species Astatotilapia burtoni serves as an important laboratory model for sexually dimorphic physiology and behavior, and also serves as an outgroup species for the explosive adaptive radiations of cichlid species in Lake Malawi and Lake Victoria. An astounding diversity of genetic sex determination systems have been revealed within the adaptive radiation of East African cichlids thus far, including polygenic sex determination systems involving the epistatic interaction of multiple, independently segregating sex determination alleles. However, sex determination has remained unmapped in A. burtoni. Here we present mapping results supporting the presence of multiple, novel sex determination alleles, and thus the presence of polygenic sex determination in A. burtoni.ResultsUsing mapping in small families in conjunction with restriction-site associated DNA sequencing strategies, we identify associations with sex at loci on linkage group 13 and linkage group 5–14. Inheritance patterns support an XY sex determination system on linkage group 5–14 (a chromosome fusion relative to other cichlids studied), and an XYW system on linkage group 13, and these associations are replicated in multiple families. Additionally, combining our genetic data with comparative genomic analysis identifies another fusion that is unassociated with sex, with linkage group 8–24 and linkage group 16–21 fused in A. burtoni relative to other East African cichlid species.ConclusionsWe identify genetic signals supporting the presence of three previously unidentified sex determination alleles at two loci in the species A. burtoni, strongly supporting the presence of polygenic sex determination system in the species. These results provide a foundation for future mapping of multiple sex determination genes and their interactions. A better understanding of sex determination in A. burtoni provides important context for their use in behavioral studies, as well as studies of the evolution of genetic sex determination and sexual conflicts in East African cichlids.Electronic supplementary materialThe online version of this article (doi:10.1186/s12864-016-3177-1) contains supplementary material, which is available to authorized users.