It is well established that the ocean is currently losing dissolved oxygen (O2) in response to ocean warming, but the long‐term, equilibrium response of O2 to a warmer climate is neither well quantified nor understood. Here we use idealized multimillennial global warming simulations with a comprehensive Earth system model to show that the equilibrium response in ocean O2 differs fundamentally from the ongoing transient response. After physical equilibration of the model (>4,000 years) under a two times preindustrial CO2 scenario, the deep ocean is better ventilated and oxygenated compared to preindustrial conditions, even though the deep ocean is substantially warmer. The recovery and overshoot of deep convection in the Weddell Sea and especially the Ross Sea after ~720 years causes a strong increase in deep ocean O2 that overcompensates the solubility‐driven decrease in O2. In contrast, O2 in most of the upper tropical ocean is substantially depleted owing to the warming‐induced O2 decrease dominating over changes in ventilation and biology. Our results emphasize the millennial‐scale impact of global warming on marine life, with some impacts emerging many centuries or even millennia after atmospheric CO2 has stabilized.