Masks are essential, especially in medical institutions, due to the global spread of illnesses and epidemics. This paper presents an unprecedented neural network called the capsule network for face mask recognition. The capsule network has proven to be most suitable for real-life image recognition, as it relies on the spatial relationship features of the image. This paper presents an adapted capsule network by adding a block for deep feature extraction. The proposed system has two phases; the first phase usesVGG16 and VGG19 as a pre-training module for the feature extractions, while the second phase is based on the Capsule network for the face mask recognition phase. Two benchmark datasets are used to test the proposed approach; Real-World Masked Face Dataset (RMFD) and Simulated Masked Face Recognition Dataset (SMFRD).The accuracy of the testing system based on RMFD data sets of CapsNet, VGG16, and VGG19 is 99.87%, 99.90%, and 99.94%, respectively. In contrast, the accuracy of CapsNet with VGG19 reaches 99.94% on the SMFD data. Comprehensive experiments demonstrate the effectiveness of the presented face mask recognition system.