Background Since the beginning of human genetic research, there are very few publications sharing insights of the negative impact of rare genetic skin diseases (RGSD) on patients’ experiences. This systematic review assessed the psychosocial implications of these conditions in terms of daily life experiences, emotional state, self-perception, and Quality of Life (QoL). Methodology A systematic review was carried out on albinism, neurofibromatosis type 1 (NF1), birthmarks and inherited ichthyosis. The PubMed, Scopus, PsycArticle, PsychInfo, Psychology and Behavioral Sciences Collection, and SOCindex databases were queried. Inclusion criteria were adult patients with one of these RGSDs. Simple descriptive statistics and qualitative content analysis were conducted to summarize the main results reported by the authors. Results Of the 9987 articles retrieved, 48 articles were included: albinism (16), NF1 (16), inherited ichthyosis (10), birthmarks (6). The majority of the studies on albinism were conducted in Africa. Twenty-seven studies quantitatively assessed diverse psychological parameters: 13 showed a significant impact of the disease on QoL, five on emotional state, two on self-representation and two others on psychiatric comorbidities. Disease severity and visibility were good predictors of QoL (except for albinism). Body image and appearance concerns were also associated with QoL and emotional state. The 19 qualitative studies highlighted recurring themes across each of these diseases: discrimination and stigma during childhood and adolescence, discomfort in social interactions, guilt of transmission, the importance of social support from family and friends, altered daily life functioning, altered romantic and sex life, limited academic and professional aspirations, lack of interest and support from the medical field, and the unpredictability of the evolution of the disease. The only two mixed-method studies in this review were unable to contribute to any inferential analyses but could corroborate some of the qualitative findings. Conclusion These results showed that RGSDs have a significant impact on different aspects of patients’ lives. This review has demonstrated that there is a real need for support systems for patients with these diseases. Such systems should be developed to provide them with necessary information and to guide them through an appropriate care pathway.