Many dioecious angiosperms are trees, which only flower after years of vegetative development and do not usually exhibit marked secondary sexual dimorphism. Nevertheless, if the genetic basis of sex determination is known, the sex of an individual can be determined using molecular markers. Here, we report that in the genus Populus sect. Populus an XY system of sex determination, which is found in P. tremula and P. tremuloides, likely re-evolved from a ZW system present in P. alba, P. adenopoda and P. qiongdaoensis. Strikingly, this new XY system is mechanistically identical to the older system found in several species of the Populus sections Tacamahaca, Aigeiros and Turanga demonstrating a remarkable example of convergent evolution. In both XY systems, male-specific inversely repeated sequences appear to silence the ARR17 gene, which functions as a sex switch, via small interfering RNAs and DNA methylation. In the ZW system, female-specific copies of ARR17 appear to regulate dioecy. With this detailed information on the genetic basis of sex determination it was possible to develop molecular markers that can be utilized to determine the sex in seedlings and non-flowering trees of different poplar species. We used the female-specific ARR17 gene to develop a sex marker for P. alba and P. adenopoda. For P. grandidentata, we employed the male-specific ARR17 inverted repeat. Finally, we summarize previously described markers for P. tremula, P. tremuloides, P. trichocarpa, P. deltoides and P. nigra. These markers can be useful for poplar ecologists, geneticists and breeders.