Unraveling the complex genetic bases and mechanisms underlying salt tolerance is of great importance for developing salt-tolerant varieties. In this study, we evaluated 42 chromosome segment substitution lines (CSSLs) carrying chromosome segments from IR64 on the genetic background of Koshihikari under salt stress. Two CSSLs, SL2007 and SL2038, produced higher plant dry weight and grain yield than did Koshihikari under the stress condition. These CSSLs also showed lower Na+ and Cl− accumulation in the leaf and whole plant at the full heading stage, which might be related to the higher grain yield and yield components. To understand the genetic control of its grain yield and yield components, a SL2007/Koshihikari F2 population was generated for quantitative trait locus (QTL) analysis. Six QTLs for grain yield and yield-related traits were detected on chromosome 2. Using near-isogenic lines (NILs) from a SL2007/Koshihikari F5 population, qSTGY2.2 was delimited to a 2.5 Mb region and novel qSTPN2 was delimited to a 0.6 Mb region. We also detected a novel QTL, qSTGF2, for grain filling, which was considered an important contributor to grain yield under salt stress in this CSSL. Our results provide insights into mechanisms conferring grain yield under salinity stress and new genetic resources for cloning and breeding.