Emerging sliding ferroelectricity (SF) holds great potential for the development of low‐energy‐cost and high‐endurance ferroelectric devices. In the van der Waals (vdWs) stacking of SF, atomic vacancies inevitably exist and gas molecules commonly stay in the interlayer, but their impact on SF is unclear. In this work, the bilayer WS2 is taken as an example and demonstrate their effect on the SF polarization and switching barrier. The sulfur vacancy (SV) is found to slightly impair polarization, but the W atoms around the SV tend to chemically adsorb O2 molecules in the vdWs gap, which can possibly further dissociate into separately chemisorbed O atoms at room temperature. The adsorbed oxygen causes the reduction of polarization and switching barrier, eventually inducing the degradation of SF properties. In addition, the adsorbed oxygen also modifies the Schottky barriers in SF‐based transistors and narrows the memory window, leading to the degradation of the devices. These effects may accumulate over time and eventually result in degraded device performance. This work provides a microscopic insight into the effect of defects/impurities on SF, favoring optimizing the performance of SF‐based devices.