The solar radiation near the surface is the main reason that affects photovoltaic power generation. Accurate ultra‐short‐term solar radiation prediction is the premise of photovoltaic power generation prediction. Here the cloud movement prediction method based on the ground‐based cloud images is presented. The cloud recognition, cloud matching, cloud area correction and cloud movement prediction are performed to predict the drift trajectory of the clouds that will block the sun. Then, using digital image technology, 13 kinds of feature information are extracted from the ground‐based cloud images. Then, these feature information are input into BP neural network, and the parameters of BP neural network are optimized by genetic algorithm. Through a large number of data training, a new ultra‐short‐term prediction model of solar radiation is established. Finally, through experimental comparison, the results show that the prediction accuracy of the model with the feature information of ground‐based cloud images can reach 96%, compared with the model without the feature information of ground‐based cloud images, the accuracy is improved by 5%. The proposed ultra‐short‐term solar radiation prediction model can effectively predict the radiation jumping process caused by cloud occlusion, and greatly improve the prediction accuracy, especially in cloudy weather.