Image watermarking is a robust solution for solving key issues like copyright protection and proof of ownership of digital data. Existing schemes of image watermarking mostly used grayscale or binary images as embedded watermarks, while only a few watermarking schemes are developed for color images. In this paper, we propose a novel robust semi-blind image watermarking scheme based on finite ridgelet transform (FRT), discrete wavelet transform (DWT), singular value decomposition (SVD), particle swarm optimization (PSO), and Arnold transform to protect copyright and verify the authenticity of color images. Firstly, the color image is converted from RGB to YCbCr color space, and the luminance component (Y) is taken into account to insert the watermark data. In this study, the principal component (PC) of the watermark image is directly inserted into the corresponding singular value of the cover image by the scaling factor to avoid the false positive problem (FPP). To further improve security, Arnold transform is applied to process the Y channel of the watermark image before inserting it in the cover image. Besides, PSO optimizes the embedding factor matrices. The qualitative evaluation factors like peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) are used to assess the visual quality, while normalized correlation coefficient (NCC) is used to assess the resemblance between the watermarked and the restored watermarked images. The performance of the proposed scheme is evaluated using geometric, non-geometric, and combinational attacks, and its comparison is performed with different image watermarking schemes to prove its robustness.