In actual application scenarios of the real-time video confidential communication, encrypted videos must meet three performance indicators: security, real-time, and format compatibility. To satisfy these requirements, an improved bitstream-oriented encryption (BOE) method based chaotic encryption for H.264/AVC video is proposed. Meanwhile, an ARM-embedded remote real-time video confidential communication system is built for experimental verification in this paper. Firstly, a 4-D self-synchronous chaotic stream cipher algorithm with cosine anti-controllers (4-D SCSCA-CAC) is designed to enhance the security. The algorithm solves the security loopholes of existing self-synchronous chaotic stream cipher algorithms applied to the actual video confidential communication, which can effectively resist the combinational effect of the chosen-ciphertext attack and the divide-and-conquer attack. Secondly, syntax elements of the H.264 bitstream are analyzed in real-time. Motion vector difference (MVD) coefficients and direct-current (DC) components in Residual syntax element are extracted through the Exponential-Golomb decoding operation and entropy decoding operation based on the context-based adaptive variable length coding (CAVLC) mode, respectively. Thirdly, the DC components and MVD coefficients are encrypted by the 4-D SCSCA-CAC, and the encrypted syntax elements are re-encoded to replace the syntax elements of the original H.264 bitstream, keeping the format compatibility. Besides, hardware codecs and multi-core multi-threading technology are employed to improve the real-time performance of the hardware system. Finally, experimental results show that the proposed scheme, with the advantage of high efficiency and flexibility, can fulfill the requirement of security, real-time, and format compatibility simultaneously.