Enhancing the Security of Vigenere and Polybius Hybrid Cryptography System using Modified Linear Congruential Generation Algorithm

Abstract

In the arena of security, various encryption algorithms are used to protect information transmitted through insecure channels. Employing a hybrid encryption algorithm provides a strong defense and complicates the decryption process for potential intruders. A hybrid cipher system that combines the Vigenere and Polybius ciphers has proven to be more secure than each cipher alone. One strategy to bolster cipher security involves generating distinct cipher text results for the same plain text and key during different instances. However, hybrid algorithm faces vulnerable to different cryptanalysis attacks such as frequency analysis attack, brute force attack and dictionary attack due to the use of repeating nature of the Vigenere key and unmodified Polybius table for encryption. To address these challenges, we propose an algorithm introducing dynamic cipher text, a random key generated through a modified linear congruential generation algorithm, and a randomized Polybius table for each character using the Randi function. Simulation results demonstrate that the proposed algorithm achieves enhanced security metrics, supporting a 96% of avalanche effect, improved confusion and diffusion properties, a mere 1.54 % average occurrence of character in the cipher text, and a 0.01839 index of coincidence value. Therefore, the proposed algorithm outperforms existing hybrid algorithms in terms of security.