Desmoid-type fibromatosis (DTF) is a rare, soft-tissue tumour. These tumours do not metastasize, but their local aggressive tumour growth and unpredictable behaviour can have a significant impact on health-related quality of life (HRQoL). Little is known about which DTF patients are particularly affected by an impaired HRQoL. The objectives of this study were to assess HRQoL among different groups of DTF patients and to investigate which socio-demographic and clinical characteristics were associated with DTF-specific HRQoL. A cross-sectional study was conducted among DTF patients from the United Kingdom and the Netherlands. HRQoL was assessed using the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire Core 30 (EORTC QLQ-C30), accompanied by the DTF-QoL to assess DTF-specific HRQoL. The scores were compared amongst subgroups, based on the socio-demographic and clinical characteristics of DTF patients. Multiple linear regression analyses with a backward elimination were conducted to identify the factors associated with DTF-specific HRQoL. A total of 235 DTF patients completed the questionnaires. Female patients, patients with more than two comorbidities, or patients who received treatment other than only active surveillance (AS) or surgery scored significantly worse on the subscales of both the EORTC QLQ-C30 and DTF-QoL. Patients that were ≥ 40 years scored significantly worse on the physical functioning scale of the EORTC QLQ-C30, while younger patients (18–39 years) scored significantly worse on several DTF-QoL subscales. Differences in the DTF-QoL subscales were found for tumour location, time since diagnosis and the presence of recurrent disease. Furthermore, treatments other than AS or surgery only, female sex, younger age and the presence of comorbidities were most frequently associated with worse scores on the DTF-QoL subscales. This study showed that (DTF-specific) HRQoL differs between groups of DTF patients. Awareness of these HRQoL differences could help to provide better, personalised care that is tailored to the needs of a specific subgroup.