Nicotine, an addictive compound found in tobacco, functions as an agonist of nicotinic acetylcholine receptors (nAChRs) in the brain. Interestingly, nicotine has been reported to act as a cognitive enhancer in both human subjects and experimental animals. However, its effects in animal studies have not always been consistent, and sex differences have been identified in the effects of nicotine on several behaviors. Specifically, the role that sex plays in modulating the effects of nicotine on discrimination learning and cognitive flexibility in rodents is still unclear. Here, we evaluated sex-dependent differences in the effect of daily nicotine administration at various doses (0.125, 0.25, and 0.5 mg/kg) on visual discrimination (VD) learning and reversal (VDR) learning in mice. In male mice, nicotine significantly improved performance in VDR, but not VD, task, while, in female mice, nicotine significantly worsened performance in the VD, but not VDR, task. Next, to investigate the cellular mechanisms that underlie the sex differences in the effects of nicotine on cognition, transcriptomic analyses were performed on prefrontal cortex tissue samples from male and female mice that had undergone VD and VDR tasks. Pathway enrichment analysis and Protein-protein interaction (P-PI) analysis using gene sets with altered gene expression found three types of effects of nicotine: those common to both sexes, those in males only, and those in females only. Decreased expression of postsynaptic-related genes in males and increased expression of innate immunity-related genes in females were identified as possible molecular mechanisms related to sex differences in the effects of nicotine on cognition in discrimination learning and cognitive flexibility.