After different consecutive waves, the pandemic phase of Coronavirus disease 2019 does not look to be ending soon for most countries across the world. To slow the spread of the COVID-19 virus, several measures have been adopted since the start of the outbreak, including wearing face masks and maintaining social distancing. Ensuring safety in public areas of smart cities requires modern technologies, such as deep learning and deep transfer learning, and computer vision for automatic face mask detection and accurate control of whether people wear masks correctly. This paper reviews the progress in face mask detection research, emphasizing deep learning and deep transfer learning techniques. Existing face mask detection datasets are first described and discussed before presenting recent advances to all the related processing stages using a well-defined taxonomy, the nature of object detectors and Convolutional Neural Network architectures employed and their complexity, and the different deep learning techniques that have been applied so far. Moving on, benchmarking results are summarized, and discussions regarding the limitations of datasets and methodologies are provided. Last but not least, future research directions are discussed in detail.