Motivated by the pursuit of both encryption efficiency and security, this paper presents a symmetric color image encryption algorithm using a novel cross–plane joint scrambling–diffusion method. To provide high–strength security, the color planes are firstly spliced horizontally and then vertically to be adequately encrypted by the joint scrambling–diffusion method. Therefore, minor changes in any pixel can cross every color plane that significantly influences the final cipher image. To achieve fast encryption speed, all employed chaotic systems have simple structures but robust properties to rapidly generate high–quality pseudorandom sequences. Additionally, the joint scrambling–diffusion method is vectorized to handle pixels in parallel for satisfactory execution efficiency. Security tests demonstrate the outstanding security performance of the proposed algorithm, with correlation coefficients and entropies of cipher images being close to 0 and greater than 7.99, respectively, and results of NPCR and UACI tests being similar to ideal cipher images, which can resist statistical, differential, exhaustive, and even chosen ciphertext/plaintext attacks. Moreover, because of the O(4W + 4H) time complexity of the novel scrambling–diffusion method, even a color image with a size of 1024 × 1024 only costs 0.26 s to be encrypted. Hence, the proposed algorithm can satisfy the efficiency and security requirements of color image encryption.