Studies on differences in brain structure and function according to sex are reported to contribute to differences in behavior and cognition. However, few studies have investigated brain structures or used tractography to investigate gender differences in pain sensitivity. The identification of tracts involved in sex-based structural differences that show pain sensitivity has remained elusive to date. Here, we attempted to demonstrate the sex differences in pain sensitivity and to clarify its relationship with brain structural connectivity. In this study, pain behavior test and brain diffusion tensor imaging (DTI) were performed in male and female rats and tractography was performed on the whole brain using fiber tracking software. We selected eight brain regions related to pain and performed a tractography analysis of these regions. Fractional anisotropy (FA) measurements using automated tractography revealed sex differences in the anterior cingulate cortex (ACC)-, prefrontal cortex (PFC)-, and ventral posterior thalamus-related brain connections. In addition, the results of the correlation analysis of pain sensitivity and DTI tractography showed differences in mean, axial, and radial diffusivities, as well as FA. This study revealed the potential of DTI for exploring circuits involved in pain sensitivity. The behavioral and functional relevance’s of measures derived from DTI tractography is demonstrated by their relationship with pain sensitivity.