The effect of B segregation at prior austenite grain boundary (PAGB) on the microstructure and toughness in the heat‐affected zone (HAZ) after high heat input welding at kJ cm−1 for Mg‐deoxidized shipbuilding steel plates is clarified. With 0.0022 wt.% B addition, B segregation occurs at PAGBs in two forms: solute B and fine Fe23(BC)6 precipitates. B segregation with a concentration of about 4 at.% decreases boundary energy by 0.13 J m−2, which reduces the nucleation density of boundary ferrites by increasing the critical activation energy of ferrite nucleation. Thus, coarse grain boundary ferrites (GBFs) and ferrite side plates (FSPs) are suppressed with film‐like allotriomorphic ferrites formed at PAGBs, and ferrite transformation temperature reduces from 712–543 to 679–541 °C. In B‐free steels, (Fe, Mn)3C particles form at PAGBs with the Mn‐depleted zone (MDZ) at around (Fe, Mn)3C, promoting GBFs and FSPs. However, (Fe, Mn)3C precipitates and MDZ are not observed at PAGBs with B addition, because B segregation delays Mn diffusion to PAGBs. Thus, GBFs and FSPs reduce, and intragranular acicular ferrites (IAFs) increase with B addition, making the dislocation density and high‐angle grain boundary density increase, which improve HAZ toughness at −40 °C from 40 to 141 J.