Loss of genetic variability is an increasing challenge in tree breeding programs due to the repeated use of a reduced number of founder genotypes. However, in almond, little is known about the genetic variability in current breeding stocks, although several cases of inbreeding depression have been reported. To gain insights into the genetic structure in modern breeding programs worldwide, marker-verified pedigree data of 220 almond cultivars and breeding selections were analyzed. Inbreeding coefficients, pairwise relatedness, and genetic contribution were calculated for these genotypes. The results reveal two mainstream breeding lines based on three cultivars: “Tuono”, “Cristomorto”, and “Nonpareil”. Descendants from “Tuono” or “Cristomorto” number 76 (sharing 34 descendants), while “Nonpareil” has 71 descendants. The mean inbreeding coefficient of the analyzed genotypes was 0.041, with 14 genotypes presenting a high inbreeding coefficient, over 0.250. Breeding programs from France, the USA, and Spain showed inbreeding coefficients of 0.075, 0.070, and 0.037, respectively. According to their genetic contribution, modern cultivars from Israel, France, the USA, Spain, and Australia trace back to a maximum of six main founding genotypes. Among the group of 65 genotypes carrying the Sf allele for self-compatibility, the mean relatedness coefficient was 0.125, with “Tuono” as the main founding genotype (24.7% of total genetic contribution). The results broaden our understanding about the tendencies followed in almond breeding over the last 50 years and will have a large impact into breeding decision-making process worldwide. Increasing current genetic variability is required in almond breeding programs to assure genetic gain and continuing breeding progress.