During thermal deformation, grain coarsening due to grain growth and grain refinement resulting from dynamic recrystallization (DRX) collectively influence the deformed grain size. To investigate the separative and comprehensive effects of the two mechanisms in the Ni-38Cr-3.8Al alloy, grain growth experiments and isothermal compression tests were conducted. Kinetics models for grain growth and DRX behaviors were established based on the experimental data, which were integrated with finite element (FE) techniques to simulate the evolution of grain size throughout the entire thermal compression process. The effects of grain coarsening and grain refinement during this process were separated and quantified based on the simulation data. The results revealed that grain coarsening predominated during the heating and holding stages, with a longer holding time and higher holding temperatures intensifying this effect. However, during the compression stage, grain coarsening and grain refinement co-existed, and their competition was influenced by deformation parameters. Specifically, grain refinement dominated at strain rates exceeding 0.1 s−1, while grain coarsening dominated at lower strain rates (<0.1 s−1) and higher deformation temperatures (>1373 K). The simulated grain sizes closely matched the experimental observations.