The poor carrier mobility of polycrystalline Mg3Sb2 at low temperatures strongly degrades the thermoelectric performance. Ionized impurities are initially thought to dominate charge carrier scattering at low temperatures. Accordingly, the increased electrical conductivity by replacing Mg with metals such as Nb is also attributed to reduced ionized impurity scattering. Recent experimental and theoretical studies challenge this view and favor the grain boundary (GB) scattering mechanism. A reduction of GB scattering improves the low‐temperature performance of Mg3(Sb, Bi)2 alloys. However, it is still elusive how these metal additions reduce the GB resistivity. In this study, Nb‐free and Nb‐added Mg3Sb2 are studied through diffraction, X‐ray absorption spectroscopy, solid‐state nuclear magnetic resonance spectroscopy, and atom probe tomography. It is shown that Nb does not enter the Mg3Sb2 matrix and remains in the metallic state. Besides, Nb diffuses along the GB forming a wetting layer, which modifies the interfacial energy and accelerates grain growth. The GB resistivity appears to be reduced by Nb‐enrichment, as evidenced by modeling the electrical transport properties. This study not only confirms the GB scattering in Mg3Sb2 but also reveals the hitherto hidden role of metallic additives on enhancing grain growth and reducing the GB resistivity.