With the development of cloud technology and the innovation of information network technology, people’s dependence on the network has gradually increased, and there are some loopholes in cloud data access. The traditional account permission model can no longer meet the conditions of cloud data access alone. If the visitor temporarily leaves the computer or goes out in an emergency, the data are likely to be leaked. Based on the importance and concern of this issue, some scholars have proposed an authentication system combined with biometric face recognition, but the traditional face recognition system has certain security risks. Such as using face pictures and videos to deceive the system, tampering with face templates, etc. Based on this, this paper proposes an encrypted face authentication system based on CNN neural network. Through the authentication of face data, the content transmission of each part is carried out in the form of ciphertext to ensure the security of information. The experimental results in this paper show that the authentication accuracy rate of DeepID is 94% when it is not encrypted, and the authentication accuracy rate decreases slightly after encryption, which is 93.3%. It is similar in other cases. When the network structure and data set remain unchanged, encryption reduces the authentication accuracy rate by 0.3%–2.4%. It can be seen that the scheme proposed in this chapter improves the system security at the cost of a smaller accuracy rate.