This study aimed to understand the impact of sex on the neurocognitive function of patients with Parkinson’s disease (PD). Ninety-four participants with idiopathic PD and 167 age-matched healthy individuals as normal controls (NCs) were recruited and underwent comprehensive neuropsychological assessments. Sex differences were found in NCs, but not in patients with PD. Among male participants, patients with PD showed worse performance on the Digit Symbol Substitution (DSS) (p < 0.001) test and Symbol Search (SS) (p < 0.001) than NCs. Among female participants, patients with PD showed worse performance on the category score of the Modified Wisconsin Card Sorting Test (p < 0.001), SS (p < 0.001), and pentagon copying (p < 0.001) than NCs. After controlling for the effects of age and years of education, Hoehn and Yahr stage was found to predict the performance of the Color Trails Test part A (βA = 0.241, pA = 0.036), Stroop Color and Word Test (β = −0.245, p = 0.036), and DSS (β = −0.258, p = 0.035) in men with PD. These results indicate the differential effect of sex on the neurocognitive function among healthy aging and PD populations. The disappearance of sex differences, which is present in healthy aging, in patients with PD suggests a gradual loss of the neuroprotective effect of estrogen after the initiation of the neurodegenerative process. This study also found mental flexibility and visuospatial function to be the susceptible cognitive domains in women with PD, while the disease severity could predict the working memory and processing speed in men with PD.