Rainbow particle image velocimetry (PIV) can restore three-dimensional (3D) velocity field of particles with a single camera, however, it requires a relatively long time to complete the reconstruction. This paper proposes a hybrid algorithm that combines the fast Fourier transform (FFT) based co-correlation algorithm and the Horn-Schunck (HS) optical flow pyramid iterative algorithm to improve the reconstruction speed. The Rankine vortex simulation experiment was performed. in which the particle velocity field was reconstructed using the proposed algorithm and the rainbow PIV method. The average endpoint error and average angular error of the proposed algorithm were roughly the same as those of the Rainbow PIV algorithm; nevertheless, the reconstruction time was 20% less. Furthermore, the effect of velocity magnitude and particle density on the reconstruction results was analyzed. In the end, the performance of the proposed algorithm was verified using real experimental single-vortex and double-vortex datasets, from which a similar particle velocity field was obtained compared with the Rainbow PIV algorithm. The results show that the reconstruction speed of the proposed hybrid algorithm was approximately one fourth faster than that of the Rainbow PIV algorithm.