To investigate the effects of small amounts of oxygen on the crystal growth process and structural changes in HfO2-based films, film was deposited on a Si substrate using HfO2 and ZrO2 targets via RF magnetron co-sputtering with small amounts of added O2. Even when the deposition was carried out without heating, the most stable monoclinic phase mainly formed at O2 partial pressures above 1 mPa, where the sputtering maintained the oxide mode in the as-deposited state. With decreasing O2 partial pressure, the amorphous component increased. During the annealing process, the metastable tetragonal or orthorhombic phase crystallized when the amorphous film was deposited at a lower O2 partial pressure of 1 mPa. The volume fraction of the metastable phase decreased abruptly at an O2 partial pressure at which the sputtering mode changed from metal mode to oxide mode. These results indicate that the O2 partial pressure during deposition have an effect on the crystal growth process and causes structural changes in the film even after the annealing process.