Health‐related quality of life (HRQOL) is reduced in Fabry disease (FD) and associated with clinical disease manifestations, but few have used Fabry‐specific severity scores to study how disease burden interferes with quality of life. We investigated how the Fabry DS3, consisting of four somatic domains and one patient‐reported item, associates with HRQOL, while also evaluating fatigue, pain and psychological distress as possible predictors. Thirty‐six adults with FD completed the Short‐form Health Survey (SF‐36), the hospital anxiety and depression scale (HADS), the brief pain inventory (BPI) and reported fatigue on a visual analog scale. Clinical data were collected from the last multidisciplinary hospital visit. Using correlation and hierarchical linear regression analyses, we examined associations between demographic, clinical and self‐reported predictors and the SF‐36 physical (PCS) and mental (MCS) component summary scores. Males scored lower than the general population in all SF‐36 domains (P < .05). General health and social functioning were reduced in females. Before including self‐reported symptom scores, DS3 showed associations with PCS (P = .009). Our fully adjusted model explained 66% of the variation in PCS, where education (P = .040) and fatigue (P = .002) retained significance. With HADS depression score (P = .001) as the sole significant factor, our regression model explained 56% of the variation in MCS. The DS3 score has implications for HRQOL in FD. Low education and fatigue represent major barriers to physical well‐being, while depression strongly influences mental quality of life. Fatigue should be recognized as an important endpoint in future FD trials. Increased efforts to diagnose and treat affective disorders are warranted.