Welding thermal cycles with heat inputs ranging from 25 to 75 kJ/cm were performed on a Gleeble 3500. The impact energy improved significantly (from 10 to 112 J), whereas the simulated coarse-grain heat-affected zone (CGHAZ) microstructure changed from lath bainite ferrite (LBF) and granular bainite ferrite (GBF) + martensite/austenite (M/A) to acicular ferrite (AF) + polygonal ferrite (PF) + M/A as the heat input increased. Simultaneously, the mean coarse precipitate sizes and the degree of V(C,N) enrichment on the precipitate surface increased, which provided favorable conditions for intragranular ferrite nucleation. The Ar3 of CGHAZ increased from 593 °C to 793 °C with increasing heat inputs; the longer high-temperature residence time inhibited the bainite transformation and promoted the ferrite transformation. As a result, acicular ferrite increased and bainite decreased in the CGHAZ. The CGHAZ microstructure was refined for the acicular ferrite segmentation of the prior austenite, and the microstructure mean equivalent diameter (MED) in the CGHAZ decreased from 7.6 µm to 4.2 µm; the densities of grain boundaries higher than 15° increased from 20.3% to 45.5% and significantly increased the impact toughness. The correlation of heat input, microstructure, and impact toughness was investigated in detail. These results may provide new ideas for the development of high welding heat input multiphase steels.