The motion of cloud over PV power station will directly cause the change of solar irradiance, which indirectly affects the prediction of minute-level PV power. Therefore, the calculation of cloud motion speed is very crucial for PV power forecasting. However, due to the influence of complex cloud motion process, it is very difficult to achieve accurate result using a single traditional algorithm. In order to improve the computation accuracy, a pattern classification and PSO optimal weights-based sky images cloud motion speed calculation method for solar PV power forecasting (PCPOW) is proposed. The method consists of two parts. Firstly, we use k-means clustering method and texture features based on Gray-Level Co-occurrence Matrix (GLCM) to classify the clouds. Secondly, for different cloud classes, we build the corresponding combined calculation model to obtain cloud motion speed. Real data recorded at Yunnan Electric Power Research Institute is used for simulation, the results show that the cloud classification and optimal combination model are effective, and the PCPOW can improve the accuracy of displacement calculation.