Capsicum is one of the most important and diverse plant taxa, widely used as a spice and vegetable worldwide, including Turkey. Germplasm characterization is an essential step for crop breeding. In the present study, we characterized the genetic diversity and population structure of a collection of 94 pepper accessions using inter-primer binding site (iPBS) retrotransposon-based markers. A total of 20 iPBS primers were used that generated 172 bands (mean = 8.6 bands/primer), of which ~92% were polymorphic in the entire germplasm collection, whereas 83%, 69%, and 80% of the bands were polymorphic within the C. annuum, C. chinense, and C. frutescens subsets, respectively. All of the taxa analyzed were clearly differentiated by the iPBS markers. The polymorphism information content of the markers ranged between 0.15 and 0.99, with an average of 0.66. Cluster analyses by different methods (UPGMA, STRUCTURE, and principal coordinate analysis) revealed a clear separation of all of the C. annuum accessions from the other pepper species, with a few subclusters observed among the latter, including groups with accessions of both C. frutescens and C. chinense. At the interspecies level, the 3 clustering methods clearly discriminated C. annuum from C. frutescens and C. chinense. No clear association was found between the iPBS-based clustering and geographical origin or fruit characteristics of the accessions. This is the first report characterizing the genetic diversity and population structure in the Turkish pepper germplasm using iPBS markers. It is expected that these data will serve as a foundation for the development of new and improved pepper varieties.