The prognostic factors in mild fibrosing interstitial lung disease (FILD) have not been established. We retrospectively attempted to identify predictive factors of annual progression in mild FILD with gender-age-physiology (GAP) stage 0 or I using logistic regression analysis, adjusting for confounding factors associated with prognosis. Annual progression was defined as more than 10% decrease in forced vital capacity (FVC) or 15% decrease in diffusing capacity of the lungs for carbon monoxide (DLCO), developing acute exacerbation, or death due to FILD. Univariate analysis showed that higher age, diagnosis of fibrotic hypersensitivity pneumonia (HP), definite usual interstitial pneumonia (UIP) pattern on CT, GAP stage I, lowest SpO2 and decrease in SpO2, and shorter walk distance in 6-minute walk test (6MWT) were significantly associated with a higher incidence of annual progression. Multivariate analysis showed that independent predictive factors of higher annual progression were diagnosis of idiopathic pulmonary fibrosis and fibrotic HP, definite UIP pattern on CT, GAP stage I, lower DLCO, lowest SpO2 and decrease in SpO2 in the 6MWT, and higher chronic pulmonary emphysema assessment test. Evaluating prognostic factors in patients with mild FILD has potential to identify patients who are suitable for early intervention with antifibrotic therapy.