Many researchers revealed that the addition of small amount of boron (B) significantly suppresses the intergranular fracture (IGF) induced by segregated phosphorous (P) along grain boundaries in iron and steels. However the details of this suppression have not been clarified yet. In the present study, the behavior of B and P during heat treatment and the mechanism of toughening caused by B in interstitial free (IF) steels containing P was examined using alpha-particle track etching (ATE), tensile testing, Auger electron spectroscopy (AES) and scanning electron microscopy (SEM). The results of ATE exhibit that the degree of grain boundary segregation of B depends on the cooling rate from 850°C and shows maximum at about 10°C/s for high B-low P steel while it shows maximum at about 555°C/s for high B-high P steel. The results of ATE also reveal that low P steel shows higher B segregation than high P steel at all cooling rates, while the difference is reduced by increasing the cooling rate. It was found that elongation at low temperatures is improved when B segregates along grain boundaries. The results of AES indicate that the grain boundary segregation of B markedly decreases the segregation of P and consequently the elongation is enhanced. Therefore, controlling of the cooling rate after recrystallization decreases the brittleness caused by P in IF steel. Segregation of B and P after heat treatment can be explained by a duplex equilibrium and non-equilibrium mechanism before and during cooling.