Pancreatic ductal adenocarcinoma (PDAC) accounts for the majority of all pancreatic cancers and is highly lethal. Focal parenchymal atrophy (FPA) of the pancreas has been reported as a characteristic imaging finding of early PDAC. Here, we reviewed 76 patients with PDAC who underwent computed tomography (CT) between 6 months and 3 years before PDAC diagnosis, as well as 76 sex- and age-matched controls without PDAC on CT examinations separated by at least 5 years. FPA was observed corresponding to the location of the subsequent tumor on pre-diagnostic CT in 14/44 (31.8%) patients between 6 months and 1 year, 14/51 (27.5%) patients between 1 and 2 years, and 9/41 (22.0%) patients between 2 and 3 years before PDAC diagnosis. Overall, FPA was more frequently observed in patients with PDAC (26/76; 34.2%) on pre-diagnostic CT than that in controls (3/76; 3.9%) (p < 0.001). FPA was observed before the appearance of cut-off/dilatation of the main pancreatic duct, suggesting that FPA might be the earliest sign of PDAC. FPA was less frequently found in tumors in the pancreatic head (3/27; 11.1%) than in those in the body (14/30; 46.7%) or tail (9/19; 47.4%). FPA may predict the subsequent PDAC diagnosis, serving as an important imaging sign for the early diagnosis of pancreatic cancer.