Some existing image encryption schemes use simple low-dimensional chaotic systems, which makes the algorithms insecure and vulnerable to brute force attacks and cracking. Some algorithms have issues such as weak correlation with plaintext images, poor image reconstruction quality, and low efficiency in transmission and storage. To solve these issues, this paper proposes an optical image encryption algorithm based on a new four-dimensional memristive hyperchaotic system (4-D MHS) and compressed sensing (CS). Firstly, this paper proposes a new 4-D MHS, which has larger key space, richer dynamic behavior and more complex hyperchaotic characteristics. The introduction of CS can reduce the image size and the transmission burden of hardware devices. The introduction of double random phase encoding (DRPE) enables this algorithm has the ability of parallel data processing and multi-dimensional coding space, and the hyperchaotic characteristics of 4-D MHS make up for the nonlinear deficiency of DRPE. Secondly, a construction method of the deterministic chaotic measurement matrix (DCMM) is proposed. Using DCMM can not only save a lot of transmission bandwidth and storage space, but also ensure good quality of reconstructed images. Thirdly, the confusion method and diffusion method proposed are related to plaintext images, which require both four hyperchaotic sequences of 4-D MHS and row and column keys based on plaintext images. The generation process of hyperchaotic sequences is closely related to the hash value of plaintext images. Therefore, this algorithm has high sensitivity to plaintext images. The experimental testing and comparative analysis results show that proposed algorithm has good security and effectiveness.