Persimmon (Diospyros kaki) is an oriental perennial woody fruit tree whose popular fruit is produced and consumed worldwide. The persimmon fruit is unique because of the hyperaccumulation of proanthocyanidins during fruit development, causing the mature fruit of most cultivars to have an astringent taste. In this study, we obtained a chromosome-scale genome assembly for ‘Youshi’ (Diospyros oleifera, 2n = 2x = 30), the diploid species of persimmon, by integrating Illumina sequencing, single-molecule real-time sequencing, and high-throughput chromosome conformation capture techniques. The assembled D. oleifera genome consisted of 849.53 Mb, 94.14% (799.71 Mb) of which was assigned to 15 pseudochromosomes, and is the first assembled genome for any member of the Ebenaceae. Comparative genomic analysis revealed that the D. oleifera genome underwent an ancient γ whole-genome duplication event. We studied the potential genetic basis for astringency development (proanthocyanidin biosynthesis) and removal (proanthocyanidin insolublization). Proanthocyanidin biosynthesis genes were mainly distributed on chromosome 1, and the clustering of these genes is responsible for the genetic stability of astringency heredity. Genome-based RNA-seq identified deastringency genes, and promoter analysis showed that most of their promoters contained large numbers of low oxygen-responsive motifs, which is consistent with the efficient industrial application of high CO2 treatment to remove astringency. Using the D. oleifera genome as the reference, SLAF-seq indicated that ‘Youshi’ is one of the ancestors of the cultivated persimmon (2n = 6x = 90). Our study provides significant insights into the genetic basis of persimmon evolution and the development and removal astringency, and it will facilitate the improvement of the breeding of persimmon fruit.