Chaos is considered as a natural candidate for encryption systems owing to its sensitivity to initial values and unpredictability of its orbit. However, some encryption schemes based on low-dimensional chaotic systems exhibit various security defects due to their relatively simple dynamic characteristics. In order to enhance the dynamic behaviors of chaotic maps, a novel 3D infinite collapse map (3D-ICM) is proposed, and the performance of the chaotic system is analyzed from three aspects: a phase diagram, the Lyapunov exponent, and Sample Entropy. The results show that the chaotic system has complex chaotic behavior and high complexity. Furthermore, an image encryption scheme based on 3D-ICM is presented, whose security analysis indicates that the proposed image encryption scheme can resist violent attacks, correlation analysis, and differential attacks, so it has a higher security level.