The grain boundary diffusion (GBD) was applied to magnetically aligned Nd-Fe-B sintered magnets on their different surfaces. The demagnetization curves show better squareness for Dy diffusing from the pole surfaces than that from the side surfaces. Kerr magnetic domain patterns on magnet surfaces in aligned and unaligned magnets were observed together with in situ optical microscope images. It indicates that the anisotropy of the GBD effect is attributed to the anisotropic texture of the grain boundary Nd-rich phase in the magnet.