Neo-tetraploid rice with high fertility is a useful germplasm for polyploid rice breeding, which was developed from the crossing of different autotetraploid rice lines. However, little information is available on the molecular mechanism underlying the fertility of neo-tetraploid rice. Here, two contrasting populations of tetraploid rice, including one with high fertility (hereafter referred to as JG) and another with low fertility (hereafter referred to as JD), were generated by crossing Huaduo 3 (H3), a high fertility neo-tetraploid rice that was developed by crossing Jackson-4x with 96025-4x, and Huajingxian74-4x (T452), a low fertility autotetraploid rice parent. Cytological, global genome sequencing-based bulked-segregant (BSA-seq) and CRISPR/Cas9 technology were employed to study the genes associated with pollen fertility in neo-tetraploid rice. The embryo sacs of JG and JD lines were normal; however, pollen fertility was low in JD, which led to scarce fertilization and low seed setting. Cytological observations displayed low pollen fertility (25.1%) and approximately 31.3 and 27.2% chromosome lagging at metaphase I and II, and 28.8 and 24.8% chromosome straggling at anaphase I and II in JD, respectively. BSA-seq of F2–3 generations and RNA-seq of F4 generation detected a common fragment, i.e., 18,915,234–19,500,000, at chromosome 7, which was comprised of 78 genes associated with fertility. Among 78 genes, 9 genes had been known to be involved in meiosis and pollen development. Two mutants ny1 (LOC_Os07g32406) and ny2 (LOC_Os07g32040) were generated by CRISPR/Cas9 knockout in neo-tetraploid rice, and which exhibited low pollen fertility and abnormal chromosome behavior. Our study revealed that two unknown genes, LOC_Os07g32406 (NY1) and LOC_Os07g32040 (NY2) play an important role in pollen development of neo-tetraploid rice and provides a new perspective about the genetic mechanisms of fertility in polyploid rice.