Electroencephalogram (EEG) plays an important role in analyzing different mental tasks and neurological disorders. Hence, it is a critical component for designing various applications, such as brain-computer interfaces, neurofeedback, etc. Mental task classification (MTC) is one of the research focuses in these applications. Therefore, numerous MTC techniques have been proposed in literary works. Although various literature reviews exist based on EEG signals for different neurological disorders and behavior analysis, there is a lack of a review of state-of-the-art MTC techniques. Therefore, this paper presents a detailed review for the MTC techniques including the classification of mental tasks and mental workload. A brief description of EEG along with its physiological and non-physiological artifacts is also presented. Further, we include information on several publicly available databases, features, classifiers, and performance metrics used in MTC studies. We implement and evaluate some of the commonly used existing MTC techniques in the presence of different artifacts and subjects, based on which the challenges and directions are highlighted for future research in MTC.