ImportanceAccurate risk prediction models using routinely measured biomarkers—eg, carbohydrate antigen 19-9 (CA19-9) and bilirubin serum levels—for pancreatic cancer could facilitate early detection of pancreatic cancer and prevent potentially unnecessary diagnostic tests for patients at low risk. An externally validated model using CA19-9 and bilirubin serum levels in a larger cohort of patients with pancreatic cancer or benign periampullary diseases is needed.ObjectiveTo assess the discrimination, calibration, and clinical utility of a prediction model using readily available blood biomarkers (carbohydrate antigen 19-9 [CA19-9] and bilirubin) to distinguish early-stage pancreatic cancer from benign periampullary diseases.Design, Setting, and ParticipantsThis diagnostic study used data from 4 academic hospitals in Italy, the Netherlands, and the UK on adult patients with pancreatic cancer or benign periampullary disease treated from 2014 to 2022. Analyses were conducted from September 2022 to February 2023.ExposuresSerum levels of CA19-9 and bilirubin from samples collected at diagnosis and before start of any medical intervention.Main Outcomes and MeasuresDiscrimination (measured by the area under the curve [AUC]), calibration, and clinical utility of the prediction model and the biomarkers, separately.ResultsThe study sample comprised 249 patients in the development cohort (mean [SD] age at diagnosis, 67 [11] years; 112 [45%] female individuals), and 296 patients in the validation cohort (mean [SD] age at diagnosis, 68 [12] years; 157 [53%] female individuals). At external validation, the prediction model showed an AUC of 0.89 (95% CI, 0.84-0.93) for early-stage pancreatic cancer vs benign periampullary diseases, and outperformed CA19-9 (difference in AUC [ΔAUC], 0.10; 95% CI, 0.06-0.14; P < .001) and bilirubin (∆AUC, 0.07; 95% CI, 0.02-0.12; P = .004). In the subset of patients without elevated tumor marker levels (CA19-9 <37 U/mL), the model showed an AUC of 0.84 (95% CI, 0.77-0.92). At a risk threshold of 30%, decision curve analysis indicated that performing biopsies based on the prediction model was equivalent to reducing the biopsy procedure rate by 6% (95% CI, 1%-11%), without missing early-stage pancreatic cancer in patients.Conclusions and RelevanceIn this diagnostic study of patients with pancreatic cancer or benign periampullary diseases, an easily applicable risk score showed high accuracy for distinguishing early-stage pancreatic cancer from benign periampullary diseases. This model could be used to assess the added diagnostic and clinical value of novel biomarkers and prevent potentially unnecessary invasive diagnostic procedures for patients at low risk.