Olive growing in Croatia has a long tradition and is of great economic and social impact. The present study includes a set of 108 tree samples (88 samples corresponding to 60 presumed cultivars and 20 trees of unnamed ones) collected from 27 groves in the entire olive growing area, and is the most comprehensive survey to be conducted in Croatia. The genetic diversity, relationships, and structures of olive plants were studied using eight microsatellite loci. All loci were polymorphic and revealed a total of 90 alleles. A total of 74 different genotypes were detected that were subjected to further diversity and genetic relationship studies. The Fitch–Margoliash tree and Bayesian analysis of population structure revealed a complex relationship between the identified olive genotypes, which were clustered into three gene pools, indicating different origins of Croatian olive germplasms. Excluding the redundant germplasms, 44 different genotypes among the sampled trees of well-known cultivars and 16 new local germplasms were identified. In addition, we provide the etymology of 46 vernacular names, which confirms that the vast majority of traditional Croatian cultivars have common and widespread names. The results presented herein underline the importance of safeguarding local cultivars and conducting continuous surveys.