In this paper, we propose a new selective encryption and decryption framework based on the start code for high efficiency video coding. There is a growing need to encrypt video information to protect video content from privacy invasion and intellectual property infringement caused by information leakage. Although encrypting an entire video is a straightforward approach, the cost to encrypting a large amount of video data is substantial, considering the resulting computational complexity. Consequently, selective encryption algorithms have been actively researched in recent years and have contributed to reducing the computational complexity. However, existing selective encryption algorithms have certain drawbacks. For instance, it is difficult to separate the video encryption algorithm from the video compression algorithm because the encryption framework is based on the syntax elements. Further, a partial reconstruction of the encrypted video bitstream is often unavoidable. To solve these problems, the proposed method encrypts encrypted the bitstream based on the start code rather than on the syntax elements. Encrypting the bitstream partially, based on the start code, makes it easy to separate the video encryption algorithm from the video compression algorithm. Furthermore, encrypting the part adjacent to the start code protects the video content, as video reconstruction using a video decoder is impossible, unless the correct start code is returned to the bitstream. The experimental results show that the proposed method reduced the encryption and decryption times by approximately 97% and 98%, respectively, compared to the encryption and decryption of the entire video bitstream. INDEX TERMS High efficiency video coding, selective encryption algorithms, start code, video encryption.