Image denoising has become a crucial task in medical ultrasound (US) imaging due to the presence of speckle or multiplicative noise and additive Gaussian noise. Recently, several denoising techniques such as adaptive wavelet thresholding & joint bilateral (AWT + JB) filter, adaptive fuzzy switching weighted mean (AFSWM) filter and median patch-based locally optimal Wiener (MPBLOW) filter have been proposed to remove the speckle noise. However, these denoising techniques were found to remove noise along with the essential parts of the actual image data which is known as over-filtering. Thereby, it reduces the accuracy of the recognition process. In this paper, a new hybrid filter technique is proposed by combining anisotropic diffusion (AD) with Butterworth band pass filter to overcome over-filtering of the image. In addition, the performance of the proposed hybrid filter and its design parameters are enhanced using the particle swarm optimization (PSO) algorithm. The simulation results show that the proposed filtering technique achieves a better denoising performance when compared with other filtering techniques in terms of peak signal-to-noise ratio (PSNR), signal-to-noise ratio (SNR), structural similarity index (SSIM) and edge preservation index (EPI). Moreover, the results validated that the proposed filtering technique using PSO achieves effective performance than using the harmony search algorithm (HSA) and other filtering techniques.