Single sample per person (SSPP) face recognition uses only a single face image of each subject in the gallery set to recognize the probe sample. Since single sample cannot provide intra-class variation information, the matching accuracy of the gallery faces with the faces captured in unconstrained video is usually low. Recently, in order to improve the accuracy, the sparse representation-based classification (SRC) technology has been extended to generic learning method, which uses prototype and variation dictionary (P+V) model for face recognition. Because the inter-class scatter between atoms in prototype dictionary is not big enough, and the intra-class scatter in constructed variation dictionary (such as posture and expression) is not rich enough, the robustness of P+V model for SSPP face recognition is poor. To solve this problem, a robust prototype dictionary and robust variation dictionary construction (RPRV) method is proposed. First, a set of atoms is obtained by dictionary learning method using gallery images and generic images. Second, some effective atoms are selected by the proposed function index method. Finally, the robust prototype dictionary (RP) and the robust variation dictionary (RV) are represented linearly using these effective atoms, respectively. The face recognition is performed according to the proposed RP+RV model. Experiment results using public datasets show that the proposed RPRV has strong robustness for the face captured under the unconstrained environment. Comparison results show that the proposed RPRV method outperform state-of-the-art SRC-based methods for SSPP face recognition.