Currently, many companies or even cities use surveillance cameras all the time, and due to the COVID-19 pandemic, many places have to limit the number of people in attendance. This paper proposes a method for people counting by gender and age in videos using deep learning techniques. The proposed method is based on a face detection and tracking approach combined with an alignment process to minimize the negative effect of the background information, considering occlusions and avoiding duplicate counting. Then, specialized Deep Neural Networks based on the EfficientNet architecture are employed for age and gender classification. Experimental results show that our method achieves satisfactory performance on people counting by age and gender, demonstrating the effectiveness of the present method.