Today, safeguarding sensitive content through encryption is crucial. This work presents a hybrid cryptosystem for images that employs both asymmetric and symmetric encryption. The asymmetric component involves applying the Diffie–Hellman protocol and the ElGamal cryptosystem to securely transmit two constants. These constants are necessary for the symmetrical aspect to generate dynamic permutations, substitution boxes, and round keys. Following an encryption process with fourteen rounds, the encrypted images are processed by an algorithm proposed to enhance entropy, a critical metric for assessing encryption quality. It increases the frequencies of the basic colors to achieve a histogram closely resembling a uniform distribution, but it increases the image size by approximately 8%. This improves the entropy values achieved by the hybrid cryptosystem, bringing them remarkably close to the ideal value of 8.0. In specific instances, the entropy values were elevated from 7.99926 to 8.0. The proposed method exhibits resilience against various attacks, including differential, linear, brute force, and algebraic attacks, as evaluated through the entropy, correlation, goodness of fit, Discrete Fourier Transform (DFT), Number of Pixels Change Rate (NPCR), Unified Average Changing Intensity (UACI), Avalanche Criteria (AC), contrast, energy, and homogeneity. Further, encrypted images are subjected to noise attacks ranging from 20% to 50% noise, including additive, multiplicative, occlusion noise, as well as the newly introduced χ2 noise. The noise damage is quantified using the proposed Similarity Parameter (SP), and a 3×3 median filter is employed to enhance the visual quality.
