To monitor the drowsiness of driver, this paper describes an efficient method by using three well defined phases. The threephases are facial features detection using Viola Jones, the eyetracking and yawning detection. Once the face is detected, the system is made illumination invariant by segmenting the skin part alone and considering only the chromatic components to reject most of the non face image backgrounds based on skin color. The tracking of eyes and yawning detection are done by correlation coefficient template matching. They can easily capture the image of the personby using a single camera in all directions. The feature vectors from each of the above phases are concatenated and a binarylinear support vector machine classifier is used to classify the consecutive frames into fatigue and nonfatigue states and sound an alarm for the former, if it is above the threshold time. Extensive real time experiments prove that the proposed method is highly efficient in finding the drowsiness and alerting the driver.