Recently, computer vision research results have supported many sectors to assist and solve problems. One of branch of the computer vision fields is biometric system. Many modalities have been implemented to depict the human characteristics. Face is one of the modalities that has been employed to recognize the human. A crucial problem of the face recognition is high dimensionality. The problem would impact on the computational performance, and even it could cause the process failure. Feature extraction is the solution to reduce the dimensionality. However, many cases have shown that feature extraction could fail as singularity problem. In this research, we proposed the improvement of the fisherface algorithm to solve the singularity problem. We have modified the singularity covariance matrix so that the matrix can be further handled and processed. The purpose of the paper is to improve the performance of the fisherface algorithm. We have verified our proposed algorithm by using the Olivetty Research Laboratory face image. We applied 7-cross validations to evaluate our proposed algorithm, the evaluation results achieved more than 92% accuracy.