The widespread use of information and communication tools today facilitates information access and highlights the significance of information and data security. In recent years, chaos-based encryption systems have emerged as a promising approach for protecting the confidentiality of transmitted images. In particular, memristor-based hyperchaotic systems have attracted significant attention because of their robustness and complexity. In this paper, we propose an image encryption model that employs a two-stage encryption method using various chaotic systems, including the logistic map, Lorenz chaotic system, and memristor-based hyperchaotic system, with AES and RSA encryption algorithms. The proposed hybrid scheme applies bit-based pixel diffusion and confusion techniques to improve the security of encrypted images. Statistical and security tests are conducted to compare the performance of the different encryption systems and algorithms and to present the measurement values obtained from the analysis. Our experimental results demonstrate the effectiveness of the proposed image encryption scheme in terms of security, speed, and reliability and provide valuable insights for the development of future chaos-based encryption systems.