Image moments, as a kind of global feature descriptor of images, have become a valuable tool for pattern recognition and image analysis. However, traditional methods are mainly used to deal with grayscale images. In this paper, we apply quaternions to fast and accurate polar harmonic Fourier moments, proposing a kind of quaternion fast and accurate polar harmonic Fourier moment (QFAPHFM) capable of handling color images. Furthermore, this paper provides a detailed analysis of the invariance of QFAPHFMs under rotation, scaling, and translation transformations. The experimental results show that QFAPHFMs exhibit excellent performance in both image reconstruction and object recognition tasks. QFAPHFMs achieve accurate image reconstruction under noiseless and noisy conditions, and demonstrate excellent recognition performance in the color-based object recognition tasks.