Background: Quality of life (QoL) as it is related with growth hormone deficiency (GHD) is a matter of controversy. Methods: We analyzed QoL in 95 children aged 8–18 years with isolated GHD (72% male) treated with growth hormone (GH). These children were compared to 190 age- and gender-matched healthy children with similar height [height <10th percentile; control group 1 (CG1)] and age- and gender-matched 285 healthy children of normal stature (control group 2: CG2). QoL was measured by the KINDL® questionnaire referring to six domains (physical well-being, emotional well-being, self-esteem, family, friends, and school). Results: QoL was significantly reduced in CG1 (effect-size 0.21) compared to CG2, while QoL was not significantly altered in children with GHD. In multiple linear regression analyses adjusted to age, gender, BMI, migration background, and socioeconomic status, decreasing height-SDS was associated with poorer QoL (especially emotional well-being), and treatment with GH was related significantly to better self-esteem. Increase of height-SDS in children treated with GH was associated positively with QoL and all its subscales except family and school. Conclusions: These findings suggest psychological consequences of short stature in children and an improvement of QoL in children treated with GH with the focus on self-esteem and emotional well-being.