As an algorithm with excellent performance, convolutional neural network has been widely used in the field of image processing and achieved good results by relying on its own local receptive fields, weight sharing, pooling, and sparse connections. In order to improve the convergence speed and recognition accuracy of the convolutional neural network algorithm, this paper proposes a new convolutional neural network algorithm. First, a recurrent neural network is introduced into the convolutional neural network, and the deep features of the image are learned in parallel using the convolutional neural network and the recurrent neural network. Secondly, according to the idea of ResNet's skip convolution layer, a new residual module ShortCut3-ResNet is constructed. Then, a dual optimization model is established to realize the integrated optimization of the convolution and full connection process. Finally, the effects of various parameters of the convolutional neural network on the network performance are analyzed through simulation experiments, and the optimal network parameters of the convolutional neural network are finally set. Experimental results show that the convolutional neural network algorithm proposed in this paper can learn the diverse features of the image, and improve the accuracy of feature extraction and image recognition ability of the convolutional neural network. INDEX TERMS Convolutional neural network; Artificial intelligence; Image recognition.