Previous work suggests sex differences in reward sensitivity. However, it remains unclear how men and women differ in the neural processes of reward-driven impulsivity. With a data set of 968 subjects (502 women) curated from the Human Connectome Project, we investigated sex differences in regional activations to reward and to punishment in a gambling task. Individual variations in reward-driven impulsivity were quantified by the difference in reaction time (RT) between reward and punishment blocks in the gambling task, as well as by a behavioral measure of delay discounting. At a corrected threshold, men and women exhibited significant differences in regional activations to reward and to punishment. Longer RTs during reward vs. punishment blocks, indicative of more cautious responding, were associated with left-hemispheric lateral prefrontal cortical activation to reward in men but not women. Steeper discounting was associated with higher activation to reward in the right-hemispheric dorsal anterior cingulate cortex and angular gyrus in women but not men. These sex differences were confirmed in slope tests. Together, the results highlight the sex-specific neural processes of reward-driven impulsivity with left-hemispheric prefrontal cortex supporting impulse control in men and right-hemispheric saliency circuit playing a more important role in diminished impulse control in women.