This paper proposes a new image encryption algorithm based on chaos map systems, hash algorithm, and Josephus permutation. The algorithm consists of chaos initialization, pixel position permutation, and pixel information diffusion. The algorithm’s initialization is generated by the original image, which has a high sensitivity to the initial value. The permutation step length is composed of Josephus permutation and gray curve permutation, which completely disturbs the pixel distribution. The diffusion process is composed of cross operation and ciphertext feedback, which breaks the strong correlation between pixels. The simulation results of the encryption algorithm are used to analyze its information entropy, the correlation between elements, and other indicators. The ciphertext image is attacked in several ways, and we analyzed its defense ability. Simulation results show that the algorithm can effectively encrypt image information and has a good defense against various attacks.