Within the domain of optical frequency comb systems operating in the mid-infrared region, extensive exploration has been undertaken regarding critical parameters, such as stabilization, coherence, or spectral tunability. Despite this, certain essential parameters remain inadequately addressed, particularly concerning light source stability at high average powers. This study explores stability limitations of an optical parametric oscillator system when scaling to several watts of average power of the idler. Notably, the highest average power reported in the 3–5 μm region, reaching 10.3 W for the idler output at 3.1 μm, is achieved. Additionally, we analyze the phase noise and beam quality of both idler and signal beams and identify the onset of higher order modes as limiting for stability at high-power operation. Finally, we estimate the free-running optical linewidth of our idler beam to be ∼300 kHz, undermining the high passive temporal stability of our source. These findings represent a significant advancement toward the realization of highly stable high-power optical frequency combs in the mid-infrared region, thereby facilitating applications previously constrained by light source average powers and quality limitations.