Inside individual cells, protein population counts are subject to molecular noise due to low copy numbers and the inherent probabilistic nature of biochemical processes. We investigate the effectiveness of proportional, integral and derivative (PID) based feedback controllers to suppress protein count fluctuations originating from two noise sources: bursty expression of the protein, and external disturbance in protein synthesis. Designs of biochemical reactions that function as PID controllers are discussed, with particular focus on individual controllers separately, and the corresponding closed-loop system is analyzed for stochastic controller realizations. Our results show that proportional controllers are effective in buffering protein copy number fluctuations from both noise sources, but this noise suppression comes at the cost of reduced static sensitivity of the output to the input signal. In contrast, integral feedback has no effect on the protein noise level from stochastic expression, but significantly minimizes the impact of external disturbances, particularly when the disturbance comes at low frequencies. Counter-intuitively, integral feedback is found to amplify external disturbances at intermediate frequencies. Next, we discuss the design of a coupled feedforward-feedback biochemical circuit that approximately functions as a derivate controller. Analysis using both analytical methods and Monte Carlo simulations reveals that this derivative controller effectively buffers output fluctuations from bursty stochastic expression, while maintaining the static input-output sensitivity of the open-loop system. In summary, this study provides a systematic stochastic analysis of biochemical controllers, and paves the way for their synthetic design and implementation to minimize deleterious fluctuations in gene product levels.