The establishment of surrogate markers to detect disability progression in persons with multiple sclerosis (PwMS) is important to improve monitoring of clinical deterioration. Optical coherence tomography (OCT) could be such a tool. However, sufficient longitudinal data of retinal neuroaxonal degeneration as a marker of disease progression exist only for PwMS with a relapsing–remitting course (RRMS) so far. In contrast, longitudinal data of retinal layers in patients with primary-progressive MS (PPMS) are inconsistent, and the association of OCT parameters with ambulatory performance in PwMS has rarely been investigated. We aimed to investigate the relative annual rates of change in retinal layers in PwMS (RRMS and PPMS) compared with healthy controls (HC) using OCT and to evaluate their association with ambulatoryfunctionalscore (AS) worsening in PPMS. A retrospective analysis of a longitudinal OCT dataset of the retinal layers of PwMS and HC from two MS centers in Germany was performed. Walking ability was measured over a standardized distance of 500 m, and changes during the observation period were categorized using the AS and the expanded disability status scale (EDSS). 61 HC with 121 eyes and 119 PwMS (PPMS: 57 patients with 108 eyes; RRMS: 62 patients with 114 eyes) were included. The median follow-up time for PwMS was 3 years. The relative annual change of pRNFL (peripapillary retinal nerve fiber layer) and INL (inner nuclear layer) was significantly different in PwMS compared with HC. RRMS and PPMS subgroups did not differ in the annual atrophy rates. In patients with PPMS, worsening of the AS was significantly associated with increased thinning of the TMV (total macular volume), GCIP (ganglion cell and inner plexiform layer), and ONPL (outer nuclear and outer plexiform layer) (all p-value < 0.05, r > 0.30). For every −0.1% decrease in the TMV, GCIP, and ONPL, the risk of a deterioration in the AS increased by 31% (hazard ratio (HR): 1.309), 11% (HR: 1.112), and 16% (HR: 1.161), respectively. In addition, worsening EDSS in PPMS was significantly associated with the relative annual atrophy rates of pRNFL, TMV, and GCIP (all p-value < 0.05). Disability progression in PPMS can be measured using OCT, and increasing annual atrophy rates of the inner retinal layers are associated with worsening ambulation. OCT is a robust and side-effect-free imaging tool, making it suitable for routine monitoring of PwMS.