Along with the recent advancements in video streaming, concerns over the security of transferred data have increased. Thus, the development of fast and reliable image encryption methodologies has become an emerging research area in the field of communications. In this paper, a systolic array-based image encryption architecture is proposed. Systolic arrays are used to apply the convolution operation, and a Lü–Chen chaotic oscillator is used to obtain a convolutional filter. To decrease resource consumption, a method to fuse confusion and diffusion processes by using systolic arrays is also proposed in this paper. The results show that the proposed method is highly secure against some differential and statistical attacks. It is also shown that the proposed method has a high speed of encryption compared to other methods.