Heroin use is closely associated with emotional dysregulation, which may explain its high comorbidity with disorders such as anxiety and depression. However, the understanding of the neurobiological etiology of the association between heroin use and emotional dysregulation is limited. Previous studies have suggested an impact of heroin on diffusivity in white matter involving the emotional regulatory system, but the specificity of this finding remains to be determined. Therefore, this study investigated the association between heroin use and diffusivity of white matter tracts in heroin users and examined whether the tracts were associated with their elevated anxiety and depression levels. A sample of 26 right-handed male abstinent heroin users (25 to 42 years of age) and 32 matched healthy controls (19 to 55 years of age) was recruited for this study. Diffusion tensor imaging data were collected, and their levels of anxiety and depression were assessed using the Hospital Anxiety and Depression Scale. Our findings indicated that heroin users exhibited higher levels of anxiety and depression, but the heroin use-associated left uncinate fasciculus was only related to their anxiety level, suggesting that association between heroin and anxiety has an incremental organic basis but that for depression could be a threshold issue. This finding improves our understanding of heroin addiction and its comorbid affective disorder and facilitates future therapeutic development.