Background: A fundamental understanding of the impact of bariatric surgery (BRS) on mechanisms of colorectal carcinogenesis is limited. For instance, studies report a reduced risk of colorectal cancer in females but not in males after BRS. We examined whether this sex-specific difference existed at the earlier polyp development stage. Methods: This retrospective cohort study included 281,417 adults from the 2012–2020 MarketScan database. We compared polyps rates on colonoscopy in four groups: post- vs. pre-BRS (treatment) to post- vs. pre-severe obesity (SO) diagnosis (control). We focused our main analysis on a propensity-matched sample that yielded a balanced distribution of covariates in our four groups (n = 9680 adults, 21.9% males). We also adjusted for important covariates. Results: Metabolic syndrome parameters improved after bariatric surgery and worsened after severe obesity diagnosis (p < 0.05). The rate of polyps was 46.7% at a median of 0.5 years pre-BRS and 47.9% at a median of 0.6 years pre-SO diagnosis. The polyps rate was 45.4% at a median (range) of 3.2 (1.0–8.5) years post-BRS. Conversely, 53.8% of adults had polyps at 3.0 (1.0–8.6) years post-SO. There was no change in the risk of colorectal polyps in males or females post- vs. pre-BRS. However, the risk of polyps was higher in males (OR = 1.32, 95% CI: 1.02–1.70) and females (OR = 1.29, 95% CI: 1.13–1.47) post- vs. pre-SO. When compared to the control group (SO), the odds ratios for colorectal polyps were lower for males and females after bariatric surgery (OR = 0.63, 95% CI: 0.44–0.90, and OR = 0.79, 95% CI: 0.66–0.96, respectively). Conclusions: Obesity is associated with an increased risk of colorectal polyps, an effect that is ameliorated after bariatric surgery. These data are relevant for studies investigating colorectal carcinogenesis mechanisms.