Real-Time Facial and eye tracking is critical in applications like military surveillance, pervasive computing, Human Computer Interaction etc. In this work, face and eye tracking are implemented by using two well-known methods, CAMSHIFT and RANSAC. In our first approach, a frontal face detector is run on each frame of the video and the Viola-Jones face detector is used to detect the faces. CAMSHIFT Algorithm is used in the real- time tracking along with Haar-Like features that are used to localize and track eyes. In our second approach, the face is detected using Viola-Jones, whereas RANSAC is used to match the content of the subsequent frames. Adaptive Bilinear Filter is used to enhance quality of the input video. Then, we run the Viola-Jones face detector on each frame and apply both the algorithms. Finally, we use Kalman filter upon CAMSHIFT and RANSAC and compare with the preceding experiments. The comparisons are made for different real-time videos under heterogeneous environments through proposed performance measures, to identify the best-suited method for a given scenario.