This single-center, retrospective study aimed to investigate the course and prognostic factors of patients with primary Sjögren syndrome-associated interstitial lung disease (pSS-ILD). We included 120 pSS patients who underwent at least two high-resolution computed tomography (HRCT) scans between 2013 and 2021. Clinical symptoms, laboratory data, HRCT findings, and pulmonary function test results were collected. Two thoracic radiologists reviewed the HRCT findings. In patients with pSS without ILD at baseline (n = 81), no development of ILD was found on follow-up (median, 2.8 years). In patients with pSS-ILD (n = 39), total disease extent, extent of coarse reticulation, and traction bronchiectasis increased on HRCT, whereas the extent of ground glass opacity (GGO) decreased at follow-up (median, 3.2 years) (each p < 0.001). In progressive group of pSS-ILD (48.7%), the extent of coarse reticulation and coarseness score of fibrosis were increased at follow-up (p < 0.05). Usual interstitial pneumonia pattern on CT (OR, 15.237) and follow-up duration (OR, 1.403) were independent risk factors for disease progression in patients with pSS-ILD. In both progressive and non-progressive pSS-ILD, GGO decreased, whereas the extent of fibrosis increased even after treatment with glucocorticoid and/or immunosuppressants. In conclusion, progression occurred in approximately half of the pSS-ILD patients with slow gradual deterioration. Our study identified a definite group of progressive pSS-ILD who did not respond to current anti-inflammatory treatment.