The simulation welding thermal cycle technique was employed to generate different sizes of prior austenite grains. Dilatometry tests, in situ laser scanning confocal microscopy, and transmission electron microscopy were used to investigate the role of prior austenite grain size on bainite transformation in low carbon steel. The bainite start transformation (Bs) temperature was reduced by fine austenite grains (lowered by about 30 °C under the experimental conditions). Through careful microstructural observation, it can be found that, besides the Hall–Petch strengthening effect, the carbon segregation at the fine austenite grain boundaries is probably another factor that decreases the Bs temperature as a result of the increase in interfacial energy of nucleation. At the early stage of the transformation, the bainite laths nucleate near to the grain boundaries and grow in a “side-by-side” mode in fine austenite grains, whereas in coarse austenite grains, the sympathetic nucleation at the broad side of the pre-existing laths causes the distribution of bainitic ferrite packets to be interlocked.