Stripe rust, caused by Puccinia striiformis f. sp. tritici (Pst), is one of the most destructive diseases of wheat (Triticum aestivum L.) worldwide. Agropyron cristatum (L.) Gaertn. (2n = 28, PPPP), one of the wild relatives of wheat, exhibits resistance to stripe rust. In this study, wheat-A. cristatum 6P disomic addition line 4844-12 also exhibited resistance to stripe rust. To identify the stripe rust resistance locus from A. cristatum 6P, ten translocation lines, five deletion lines and the BC2F2 and BC3F2 populations of two wheat-A. cristatum 6P whole-arm translocation lines were tested with a mixture of two races of Pst in two sites during 2015–2016 and 2016–2017, being genotyped with genomic in situ hybridization (GISH) and molecular markers. The result indicated that the locus conferring stripe rust resistance was located on the terminal 20% of 6P short arm’s length. Twenty-nine 6P-specific sequence-tagged-site (STS) markers mapped on the resistance locus have been acquired, which will be helpful for the fine mapping of the stripe rust resistance locus. The stripe rust-resistant translocation lines were found to carry some favorable agronomic traits, which could facilitate their use in wheat improvement. Collectively, the stripe rust resistance locus from A. cristatum 6P could be a novel resistance source and the screened stripe rust-resistant materials will be valuable for wheat disease breeding.