Lonely individuals may exhibit dysfunction, particularly with respect to social empathy and self-efficacy. White matter (WM) structures related to loneliness have not yet been identified. We investigated the association between regional WM density (rWMD) using the UCLA Loneliness Scale in 776 healthy young students aged 18–27 years old. Loneliness scores were negatively correlated with rWMD in eight clusters: the bilateral inferior parietal lobule (IPL), right anterior insula (AI), posterior temporoparietal junction (pTPJ), left posterior superior temporal sulcus (pSTS), dorsomedial prefrontal cortex (dmPFC), and rostrolateral prefrontal cortex (RLPFC). The bilateral IPL, right AI, left pSTS, pTPJ, and RLPFC were strongly associated with Empathy Quotient (EQ), whereas the bilateral IPL, right AI, left pTPJ, and dmPFC were associated with General Self-Efficacy Scale (GSES) score. The neural correlates of loneliness comprise widespread reduction in WMD in areas related to self- and social cognition as well as areas associated with empathy and self-efficacy.