Enamel is the outermost layer of the tooth that protects it from invasion. In general, an acidic environment accelerates tooth demineralization, leading to the formation of cavities. Scanning electron microscopy (SEM) is conventionally used as an in vitro tool for the observation of tooth morphology changes with acid attacks. Yet, SEM has intrinsic limitations for the potential application of in vivo detection in the early demineralization process. In this study, a high-resolution optical coherence tomography (OCT) system with the axial and transverse resolutions of 2.0 and 2.7 μm in teeth has been utilized for characterizing the effect of the acidic environment (simulated by phosphoric acid) on the enamel topology. The scattering coefficient and the surface roughness of enamel can be directly derived from the OCT results, enabling a quantitative evaluation of the topology changes with demineralization. The dynamic process induced by the acid application is also recorded and analyzed with OCT, depicting the evolution of the demineralization process on enamel. Notably, the estimated enamel scattering coefficient and surface roughness significantly increase with the application time of acid and the results illustrate that the values of both parameters after demineralization are significantly larger than those obtained before the demineralization, illustrating both parameters could be effective to differentiate the healthy and demineralized teeth and determine the severity. The obtained results unambiguously illustrate that demineralization of the tooth surface can be successfully detected by OCT and further used as an indicator of early-stage cavity formation.