This study discusses the reliability analysis of voice recognition security using the deep learning convolutional neural network (CNN) algorithm. The CNN algorithm has learning advantages in that it is safer, faster, and more accurate. CNN also can solve user identification problems in large amounts of data. The measured voice input is ten types of user's voice with the number of iterations of 6000, 12000, and 15000 sound files. Furthermore, voice extraction features are performed to recognize conversations and retain information that is very much needed. After that, the voice file iteration data is trained to register the user's voice so that a trained model is obtained. These results measure performance (confusion matrix) to analyze the actual value compared to the predicted value in the CNN algorithm. The results obtained are that the best accuracy is obtained at 15000 sound file iterations, 96.87%, 12000 sound file iterations get 96.30%, and 6000 sound file iterations get 95.77%. CNN's performance data shows that 15000 iterations of voice files produce high accuracy. Voice recognition security helps provide high security and maintain the privacy of one's identity.