If successfully developed, calorimetry at tens of GPa of pressure could help characterize phase transitions in materials such as high-pressure minerals, metals, and molecular solids. Here, we extend alternating-current calorimetry to 9 GPa and 300 K in a diamond anvil cell and use it to study phase transitions in H2O. In particular, water is loaded into the sample chambers of diamond-cells, along with thin metal heaters (1 μm-thick platinum or 20 nm-thick gold on a glass substrate) that drive high-frequency temperature oscillations (20 Hz to 600 kHz; 1 to 10 K). The heaters also act as thermometers via the third-harmonic technique, yielding calorimetric data on (1) heat conduction to the diamonds and (2) heat transport into substrate and sample. Using this method during temperature cycles from 300 to 200 K, we document melting, freezing, and proton ordering and disordering transitions of H2O at 0 to 9 GPa, and characterize changes in thermal conductivity and heat capacity across these transitions. The technique and analysis pave the way for calorimetry experiments on any non-metal at pressures up to ∼100 GPa, provided a thin layer (several μm-thick) of thermal insulation supports a metallic thin-film (tens of nm thick) Joule-heater attached to low contact resistance leads inside the sample chamber of a diamond-cell.