Introduction/Background: Chemoradiotherapy (CRT) followed by durvalumab, an immune checkpoint inhibitor, is the standard treatment for locally advanced non-small-cell lung cancer (NSCLC). Interstitial lung disease (ILD) is a life-threatening toxicity caused by these treatments; however, risk factors for the ILD have not yet been established. Interstitial lung abnormalities (ILAs) are computed tomography (CT) findings which manifest as minor interstitial shadows. We aimed to investigate whether ILAs could be risk factors for grade-two or higher ILD during durvalumab therapy. Patients and Methods: Patients with NSCLC who received durvalumab after CRT from July 2018 to June 2021 were retrospectively enrolled. We obtained patient characteristics, laboratory data, radiotherapeutic parameters, and chest CT findings before durvalumab therapy. Results: A total of 148 patients were enrolled. The prevalence of ILAs before durvalumab treatment was 37.8%. Among 148 patients, 63.5% developed ILD during durvalumab therapy. The proportion of patients with grade-two or higher ILD was 33.8%. The univariate logistic regression analysis revealed that older age, high dose-volume histogram parameters, and the presence of ILAs were significant risk factors for grade-two or higher ILD. The multivariate analysis showed that ILAs were independent risk factors for grade-two or higher ILD (odds ratio, 3.70; 95% confidence interval, 1.69–7.72; p < 0.001). Conclusions: We showed that pre-existing ILAs are risk factors for ILD during durvalumab treatment after CRT. We should pay attention to the development of grade-two or higher ILD during durvalumab treatment in patients with ILAs.