The task of recognition and classification of driver's fatigue and drowsy face expression is a challenging task. There are several face parts involved in determining the fatigue and drowsiness levels of the driver. The face parts involved are eyes and mouths which portray different face expressions such as mouth open when yawning and shut or open eyes. Hence, it is very important to recognize the facial expression of the drivers based on these features. Therefore, this paper proposed the use of Shape Features for extracting the face features which are left eyes, right eyes and mouth. Next, Support Vector Machine (SVM) is deployed for the purposed of classification. Experiments have been conducted on the real-time images and promising result has achieved. The average classification accuracy achieved is 83.33%. Findings from this study is believed to expand the research towards building a fatigue and drowsy model for assisting driver in reducing traffic road accident.