Variants in the ABCA4 gene are associated with a spectrum of inherited retinal diseases (IRDs), most prominently with autosomal recessive (ar) Stargardt disease (STGD1) and ar cone-rod dystrophy. The clinical outcome to a large degree depends on the severity of the variants. To provide an accurate prognosis and to select patients for novel treatments, functional significance assessment of nontruncating ABCA4 variants is important. We collected all published ABCA4 variants from 3,928 retinal dystrophy cases in a Leiden Open Variation Database, and compared their frequency in 3,270 Caucasian IRD cases with 33,370 non-Finnish European control individuals. Next to the presence of 270 protein-truncating variants, 191 nontruncating variants were significantly enriched in the patient cohort. Furthermore, 30 variants were deemed benign. Assessing the homozygous occurrence of frequent variants in IRD cases based on the allele frequencies in control individuals confirmed the mild nature of the p.[Gly863Ala, Gly863del] variant and identified three additional mild variants (p.(Ala1038Val), c.5714+5G>A, and p.(Arg2030Gln)). The p.(Gly1961Glu) variant was predicted to act as a mild variant in most cases. Based on these data, in silico analyses, and American College of Medical Genetics and Genomics guidelines, we provide pathogenicity classifications on a five-tier scale from benign to pathogenic for all variants in the ABCA4-LOVD database.