The ABCA4 gene is one of the most common disease-causing genes of inherited retinal degeneration. In this study, we report different phenotypes of ABCA4-associated retinal dystrophies in the Taiwanese population, its clinical progression, and its relationship with genetic characteristics. Thirty-seven subjects were recruited and all patients underwent serial ophthalmic examinations at a single medical center. Fundus autofluorescence (FAF) images were quantified for clinical evaluation, and panel-based next-generation sequencing testing was performed for genetic diagnosis. Visual preservation, disease progression, and genotype–phenotype correlation were analyzed. In this cohort, ABCA4-associated retinal degeneration presented as Stargardt disease 1 (STGD1, 62.16%), retinitis pigmentosa (32.43%), and cone-rod dystrophy (5.41%). STGD1 could be further divided into central and dispersed types. In each phenotype, the lesion areas quantified by FAF increased with age (p < 0.01) and correlated with poorer visual acuity. However, three patients had the foveal sparing phenotype and had relatively preserved visual acuity. Forty-two ABCA4 variants were identified as disease-causing, with c.1804C>T (p.Arg602Trp) the most frequent (37.84%). Patients with a combination of severe/null variants could have more extensive phenotypes, such as arRP and dispersed STGD1. This is the first cohort study of ABCA4-associated retinal degeneration in Taiwan with wide spectrums of both genotypic and phenotypic characteristics. An extremely high prevalence of c.1804C>T, which has not been reported in East Asia before, was noted. The extensiveness of retinal involvement might be regarded as a spectrum of ABCA4-associated retinal dystrophies. Different types of genetic variations could lead to distinctive phenotypes, according to the coding impact of variants.