Background: Despite the importance of psychosis as a comorbidity of temporal lobe epilepsy (TLE), the underlying neural mechanisms are still unclear. We aimed to investigate abnormalities specific to psychosis in TLE, using diffusion MRI parameters and graph-theoretical network analysis. Material and Methods: We recruited 49 patients with TLE (20 with and 29 without interictal schizophrenia-like psychosis) and 42 age-/gender-matched healthy controls. We performed 3-tesla MRI scans including 3D T1-weighted imaging and diffusion tensor imaging in all participants. Among the three groups, fractional anisotropy (FA), mean diffusivity (MD), and global network metrics were compared by analyses of covariance. Regional connectivity strength was compared by network-based statistics. Results: Compared to controls, TLE patients showed significant temporal and extra-temporal changes in FA, and MD, which were more severe and widespread in patients with than without psychosis. We observed distinct differences between TLE patients with and without psychosis in the anterior thalamic radiation (ATR), inferior fronto-occipital fasciculus (IFOF), and inferior longitudinal fasciculus (ILF). Similarly, for network metrics, global, and local efficiency and increased path length were significantly reduced in TLE patients compared to controls, but with more severe changes in TLE with psychosis than without psychosis. Network-based statistics detected significant differences between TLE with and without psychosis mainly involving the left limbic and prefrontal areas. Conclusion: TLE patients with interictal schizophrenia-like psychosis showed more widespread and severe white matter impairment, involving the ATR, IFOF and ILF, as well as disrupted network connectivity, particularly in the left limbic and prefrontal cortex, than patients without psychosis.