It is not known if the degrees of improvement in oxygenation obtained by CPAP can predict clinical outcomes in patients with COVID-19 pneumonia. This was a retrospective study conducted on patients with severe COVID-19 pneumonia treated with CPAP in three University hospitals in Milan, Italy, from March 2020 to March 2021. Arterial gas analysis was obtained before and 1 h after starting CPAP. CPAP failure included either death in the respiratory units while on CPAP or the need for intubation. Two hundred and eleven patients (mean age 64 years, 74% males) were included. Baseline median PaO2, PaO2/FiO2 ratio (P/F), and the alveolar-arterial (A-a) O2 gradient were 68 (57–83) mmHg, 129 (91–179) mmHg and 310 (177–559) mmHg, respectively. Forty-two (19.9%) patients died in the respiratory units and 51 (24.2%) were intubated. After starting CPAP, PaO2/FiO2 increased by 57 (12–113; p < 0.001) mmHg, and (A-a) O2 was reduced by 68 (−25–250; p < 0.001) mmHg. A substantial overlap of PaO2, P/F, and A-a gradient at baseline and during CPAP was observed in CPAP failures and successes; CPAP-associated improvements in oxygenation in both groups were similar. In conclusion, CPAP-associated improvements in oxygenation do not predict clinical outcomes in patients with severe COVID-19 pneumonia.