The grain yield is closely associated with spikelet fertility in rice (Oryza sativa L.) under high temperatures, and nitrogen (N) plays a crucial role in yield formation. To investigate the effect of panicle N application on yield formation under high temperatures at the panicle initiation stage, two rice varieties [Liangyoupeijiu (LYPJ, heat susceptible) and Shanyou63 (SY63, heat tolerant)] were grown and exposed to high daytime temperature (HT) and control temperature (Control) during the panicle initiation stage. Low (LPN) and high (HPN) panicle N applications were conducted. HT markedly decreased the yields by 87% at LPN and 48% at HPN in LYPJ and 31% at LPN and 36% at HPN in SY63. The decrease in grain yield under HT was primarily attributed to the decline in spikelet fertility, HPN increased spikelet fertility. HT resulted in the abnormal development of anthers, which included disordered, enlarged, and broken anther wall layers, degraded and irregularly shaped microspores, delayed tapetum degradation, less vacuolated microspores per locule, abnormal and aborted pollen grains; however, HPN improved the development of anthers under HT, particularly in LYPJ. A high rate of evapotranspiration resulted in an approximately 1°C decrease in panicle temperatures at HPN compared with that at LPN in both varieties under HT. Overall, these results demonstrate that the increased panicle N application favors normal anther development in LYPJ by decreasing the panicle temperature, which results in high pollen viability and spikelet fertility, and consequently less yield loss under HT.