Heilongjiang Province is a significant region for grain production and serves as a crucial commodity grain production base in China. In recent years, due to the threat of declining cropland quality and quantity, coupled with the increasingly prominent demand for grain, there is an urgent need to enhance rice yields in Heilongjiang Province. It is imperative to accurately identify the gaps between actual and potential grain yields and effectively implement yield-enhancing measures in regions with significant yield gaps. This study aimed to determine the rice reproductive periods of Heilongjiang Province for 2000, 2010, and 2020, estimate the rice actual yields using the Vegetation Photosynthesis Model (VPM), simulate the rice potential yields based on the Global Agro-Ecological Zones (GAEZ) Model, and then identify the rice yield gaps at the pixel level by calculating the rice absolute yield gap (AYG) and relative yield gap (RYG). Additionally, yield-enhancing measures were proposed for regions with significant yield gaps. The results were as follows. (1) The rice reproductive periods of Heilongjiang Province for 2000, 2010, and 2020 were determined as days 153~249, days 145~249, and days 137~249. (2) The mean rice actual yield and potential yields decreased by 1222 and 5941 kg ha−1 during the 2000–2020 period, respectively, and the total actual and potential production increased by 3.75 and 1.70 million tons in Heilongjiang Province, respectively. (3) The rice AYG and RYG in the Sanjiang Plain region, such as Jixi City, Hegang City, and Jiamusi City were relatively large compared to other regions for the three years, and the rice yield gaps continued to decrease during the 2000–2020 period. (4) With regard to the Sanjiang Plain region with a large rice yield gap, this study proposes measures to narrow the rice yield gap by establishing ecological protection forests on cropland, transforming low- and middle-yielding fields, increasing agricultural science and technology inputs, selecting better rice cultivars, etc., which are important for ensuring food security.