Background Stroke survivors are known to have poorer health-related quality of life (HRQoL) than the general population, but less is known about characteristics associated with HRQoL decreasing through time following a stroke. This study aims to examine how in-hospital frailty is related to HRQoL from 3 to 18 months post stroke. Method Six hundred twenty-five participants hospitalised with stroke were included and followed up at 3 and/or 18 months post stroke. Stroke severity was assessed the day after admission with the National Institutes of Health Stroke Scale (NIHSS). A modified Fried phenotype was used to assess in-hospital frailty; measures of exhaustion, physical activity, and weight loss were based on pre-stroke status, while gait speed and grip strength were measured during hospital stay. HRQoL at 3- and 18-months follow-up were assessed using the five-level version of the EuroQol five-dimensional descriptive system (EQ-5D-5L) and the EuroQol visual analogue scale (EQ-5D VAS). We conducted linear mixed effect regression analyses unadjusted and adjusted for sex, age, and stroke severity to investigate the association between in-hospital frailty and post-stroke HRQoL. Results Mean (SD) age was 71.7 years (11.6); mean NIHSS score was 2.8 (4.0), and 263 (42.1%) were female. Frailty prevalence was 10.4%, while 58.6% were pre-frail. The robust group had EQ-5D-5L index and EQ-5D VAS scores at 3 and 18 months comparable to the general population. Also at 3 and 18 months, the pre-frail and frail groups had significantly lower EQ-5D-5L indices than the robust group (p <  0.001), and the frail group showed a larger decrease from 3 to 18 months in the EQ-5D-5L index score compared to the robust group (− 0.056; 95% CI − 0.104 to − 0.009; p = 0.021). There were no significant differences in change in EQ-5D VAS scores between the groups. Conclusion This study on participants mainly diagnosed with mild strokes suggests that robust stroke patients have fairly good and stable post-stroke HRQoL, while post-stroke HRQoL is impaired and continues to deteriorate among patients with in-hospital frailty. This emphasises the importance of a greater focus on frailty in stroke units. Trial registration ClinicalTrials.gov (NCT02650531).