Many functional materials are designed at the multiscale level. To properly simulate their physical properties, large and sophisticated computer models capable of replicating the microstructure with nm-level accuracy are required. This is the case for permanent magnets, for which there is a long-standing problem of a significant offset between simulated and experimental coercivity. To overcome this problem and resolve the Brown paradox, we developed a novel approach to construct large-scale finite element models based on the microstructure tomography. It was applied to ultrafine-grained Nd-Fe-B magnets for which, besides the shape, size, and packing of the grains, we reconstructed individual regions of thin intergranular phase separated by triple junctions. Such micromagnetic model managed to reproduce not only experimental coercivity but also its mechanism according to the angular dependence of coercivity. Furthermore, a remarkable role of thin triple junctions as nucleation centers for magnetization reversal was revealed. Proposed digital twins of Nd-Fe-B permanent magnets can assist their optimization towards an ultimate coercivity, while the developed tomography-based approach can be adapted to a wide range of polycrystalline materials.