For years, the main emphasis of industrial pepper production has been the creation of large-fruited high-yielding varieties. This has led to the exploitation of pepper (Capsicum annuum L.) varieties with a significantly narrow genetic background. The Balkan region has a range of valuable local landraces that are noticeably decreasing in prevalence and may be lost. In this study, we utilized Tomato Analyzer (TA) to characterize the intra-and intervarietal type fruit diversity using 50 different fruit shape, size, and color descriptors of a Balkan pepper collection. The collection comprised of 168 diverse accessions and collected from 62 locations and 31 districts in six different Balkan countries. Fruit shape was the main trait used for visual accession grouping into 5 different varietal groupings of elongated, round, conical, bell, and pumpkin shape, respectively. Beyond visual grouping, hierarchical cluster analysis grouped the evaluated accessions into eight distinct clusters based on fruit shape, size, surface, and color. In total, 47 TA descriptors were identified to contribute to the total variation, with first two components explaining 54% of variation, and 90% of variation contained in 12 components. Among TA descriptors, fruit size and proximal/distal fruit end shape contributed to component 1 variation whereas fruit perimeter, area and color contributed to component 2. We hope that this research will assist pepper breeding and genetic resources communities to better understand Balkan pepper fruit diversity and develop pepper varieties with desirable fruit traits. Quantification of fruit diversity could be crucial for further investigation into the genetic determinants of fruit shape and size by a genome-wide association study.