Background: There are only a few reports on the nutritional status and mortality of patients with idiopathic pulmonary fibrosis (IPF). As such, this study aims to investigate the relationship between controlling nutritional status (CONUT) and the mortality of elderly patients with IPF. Methods: A total of 170 IPF patients aged ≥65 years old who visited the rehabilitation department of our hospital between July 2014 and July 2021 (mean age: 75.7 ± 6.3 years, sex (male/female): 138/32, %FVC: 78.3 ± 18.3%) were retrospectively analyzed. The Kaplan–Meier method and log-rank test were applied. Furthermore, using a Cox proportional hazards model with multivariate analysis, we analyzed the relationship between all-cause mortality and baseline characteristics including CONUT. Results: Based on the CONUT score, the normal group included 101 cases, the mild group included 58 cases, the moderate group included 11 cases, and the severe group had 0 cases. There were 49 cases of all-cause mortality events, suggesting that the mortality of the moderate group was significantly poorer than that of the normal and mild groups (p < 0.05). Furthermore, multivariate analysis identified GAP stage (HR: 5.972, 95%CI: 2.901~12.291, p < 0.0001), mMRC scale (HR: 0.615, 95%CI: 0.389~0.971, p = 0.009), and CONUT (HR: 2.012, 95%CI: 1.192~3.395, p = 0.037) as factors significantly influencing mortality. Conclusions: Severe malnutrition was not observed in elderly patients with IPF. Moderate malnutrition was associated with a significantly higher risk of all-cause mortality, suggesting that CONUT is an important indicator for predicting mortality.