Background Neurodegeneration plays an important role in permanent disability in multiple sclerosis (MS). Objective To determine whether progressive neurodegeneration occurs in MS eyes without clinically-evident inflammation. Methods Retinal nerve fiver layer thickness (RNFLT) and ganglion cell-inner plexiform layer thickness (GCIPT) were measured using Cirrus optical coherence tomography (OCT) in 133 relapsing-remitting MS (RRMS) patients (149 no-ON, 97 ON eyes, last optic neuritis (ON) ≥6 months). 93 patients were scanned at two visits. Percents of abnormal GCIPT vs RNFLT (<5% of machine norms) in cross-sectional data were compared. Relations between RNFLT/GCIPT and MS duration (cross-sectional) and follow-up time (longitudinal) were assessed. Results GCIPT was abnormal in more eyes than RNFLT (27% vs 16% p=0.004 in no-ON, 82% vs 72% p=0.007 in ON). RNFLT and GCIPT decreased with MS duration by −0.49 µm/yr (p=0.0001) and −0.36 (p=0.005) for no-ON; −0.52 (p=0.003) and −0.41 (p=0.007) for ON. RNFLT and GCIPT decreased with follow-up time by −1.49 µm/yr (p<0.0001) and −0.53 (p=0.004) for no-ON, −1.27 (p=0.002) and −0.49 (p=0.04) for ON. Conclusions In RRMS eyes without clinically-evident inflammation, progressive loss of RNFLT and GCIPT occurred, supporting the need for neuroprotection in addition to suppression of auto-immune responses and inflammation.