Unconventional ferroelectricity exhibited by hafnia-based thin films, robust at nanoscale sizes, presents tremendous opportunities in nanoelectronics. However, the exact nature of polarization switching remains controversial. We investigated La0.67Sr0.33MnO3/Hf0.5Zr0.5O2 capacitor interfaced with various top electrodes while in situ electrical biasing using atomic resolution microscopy with direct oxygen imaging, as well as synchrotron nanobeam diffraction. When the top electrode is oxygen reactive, we clearly show reversible oxygen vacancy migration with electrodes being the source and sink of oxygen, and the dielectric layer acting as a fast conduit at millisecond timescales. With non-reactive top electrodes and at longer time scales (seconds), the dielectric layer also acts as an oxygen source/sink. Our results show that ferroelectricity in hafnia-based thin films is unmistakably intertwined to oxygen voltammetry.