Neuroretina and retinal pigment epithelium (RPE) are differentiated from the progenitors in optic vesicles, but it is unclear when and how the two lineages are segregated. Manipulation of chick embryos reveals that the early anteroventral optic vesicle is crucial for neuroretinal development, but the molecular mechanism is unclear. Homeodomain transcription factor Six3 is required for neuroretinal specification and is dispensable for RPE formation, but the cell fates of Six3-deficient progenitors and the origins of remnant RPE are unknown. Here, we performed lineage tracing of Six3-Cre positive cells in wild-type and Six3-deficient mouse embryos. Six3-Cre positive progenies were found in a population of progenitors in the anteroventral optic pits/vesicles starting at E8.5, and were found in neuroretina, optic stalk, ventral forebrain, but not RPE, at E10.5. Six3-deletion in the small population of progenitors at E8.5 was sufficient to cause rostral expansion of Wnt8b and drastic reduction of Fgf8/MAPK signaling, ablating neuroretinal specification without affecting RPE. Lineage tracing revealed Six3-deficient progenitors at E8.5 were eventually lost and the remnant RPE was derived from Six3-Cre negative cells. Thus, Six3 in a small population of progenitors expressing Six3-Cre at E8.5 is required for neuroretinal specification via regulating cell signaling and survival in mice.