The solid electrolyte interphase (SEI) on lithium metal anodes (LMA) plays a critical role in affording a long lifespan required for aprotic lithium‐oxygen (Li–O2) batteries. Nevertheless, the crossover of oxygen from the cathode to the anode, an inevitable phenomenon for most of the current Li–O2 batteries, and its effects on the formation and operation of SEI on LMA remain less explored. In this work, a mechanistic study of the SEI formation at a model Cu/dimethyl sulfoxide (DMSO) interface in the presence of oxygen is presented. Direct spectroscopic evidence coupled with theoretical calculation reveals that oxygen can alter the SEI formation pathway and result in distinct SEI properties. Specifically, oxygen can inhibit the fission of the C–S bond of DMSO solvent and therefore reduce the formation of unstable SEI components (e.g., C≡C species) and volatile products (e.g., C2H6 and H2). Thus, the SEI formed under oxygen is more uniform and of less voids, and enables improved electrochemical performance of LMA. This work presents new insights into the oxygen crossover effects on SEI chemistry and is beneficial for designing better LMA/electrolyte interface for future Li–O2 batteries.