We successfully demonstrate low-frequency noise suppression in the resonant frequency fluctuations of a cavity-embedded Cooper pair transistor (cCPT) driven at single-photon occupancy. In particular, we report a reduction in the resonant frequency fluctuations caused by the internal charge noise over a bandwidth of ∼1.4 kHz when the cavity is driven at an average photon number n = 10, and a bandwidth of 11 Hz for average n = 1. The gate-dependent tunability of the cCPT allows us to implement a feedback-scheme, derived from the Pound-Drever-Hall locking technique. This reduces fluctuations due to intrinsic charge-noise, which otherwise interferes with the cCPT's operation as a near quantum-limited electrometer. We believe our technique can be generalized to achieve frequency stabilization in tunable microwave resonators that play a vital role in today's quantum computing architecture, thereby moderating the limitations in detection caused by the intrinsic 1/f -noise on such circuit devices. The work discusses the various aspects relating to the operation of a fully functional feedback loop down to the single-photon level.