Rice lines derived from wild species and mutants can serve as a good resource for favorable alleles for heat tolerance. In all, 48 stable lines including 17 KMR3/O. rufipogon introgression lines (KMR3 ILs), 15 Swarna/O. nivara ILs (Swarna ILs) along with their parents, Nagina 22 (N22) and its 4 EMS induced mutants and 7 varieties were evaluated for heat tolerance under irrigated conditions under field in two seasons, wet season 2012 using poly cover house method and dry season 2013 using late sown method. Spikelet fertility (SF), yield per plant (YP) and heat susceptibility index (HSI) for these two traits were considered as criteria to assess heat tolerance compared to control. Four KMR3 ILs and eight Swarna ILs were identified as heat tolerant based on SF and YP and their HSIs in both wet and dry seasons. S-65 and S-70 showed low SF and high YP consistently in response to heat in both seasons. We provide evidence that SF alone may not be the best criterion to assess heat tolerance and including YP is important as lines with low SF but high YP and vice versa were identified under heat stress. Out of 49 SSR markers linked to spikelet fertility, 18 were validated for five traits. RM229 in wet season and RM430 and RM210 in dry season were significantly associated with both SF and its HSI under heat stress. RM430 was also significantly associated with both YP and its HSI in dry season. Thirty two candidate genes were identified close to nine markers associated with traits under heat stress.