This paper presents a novel five-dimensional three-leaf chaotic attractor and its application in image encryption. First, a new five-dimensional three-leaf chaotic system is proposed. Some basic dynamics of the chaotic system were analyzed theoretically and numerically, such as the equilibrium point, dissipative, bifurcation diagram, plane phase diagram, and three-dimensional phase diagram. Simultaneously, an analog circuit was designed to implement the chaotic attractor. The circuit simulation experiment results were consistent with the numerical simulation experiment results. Second, a convolution kernel was used to process the five chaotic sequences, respectively, and the plaintext image matrix was divided according to the row and column proportions. Lastly, each of the divided plaintext images was scrambled with five chaotic sequences that were convolved to obtain the final encrypted image. The theoretical analysis and simulation results demonstrated that the key space of the algorithm was larger than 10150 that had strong key sensitivity. It effectively resisted the attacks of statistical analysis and gray value analysis, and had a good encryption effect on the encryption of digital images.