Alterations in the retinal oxygen saturation (sO2) and oxygen consumption are associated with nearly all blinding diseases. A technology that can accurately measure retinal sO2 has the potential to improve ophthalmology care significantly. Recently, visible-light optical coherence tomography (vis-OCT) showed great promise for noninvasive, depth-resolved measurement of retinal sO2 as well as ultra-high resolution anatomical imaging. We discovered that spectral contaminants (SC), if not correctly removed, could lead to incorrect vis-OCT sO2 measurements. There are two main types of SCs associated with vis-OCT systems and eye conditions, respectively. Their negative influence on sO2 accuracy is amplified in human eyes due to stringent laser power requirements, eye motions, and varying eye anatomies. We developed an adaptive spectroscopic vis-OCT (Ads-vis-OCT) method to iteratively remove both types of SCs. We validated Ads-vis-OCT in ex vivo bovine blood samples against a blood-gas analyzer. We further validated Ads-vis-OCT in 125 unique retinal vessels from 18 healthy subjects against pulse-oximeter readings, setting the stage for clinical adoption of vis-OCT.