Background and objective: Post-endoscopic retrograde cholangiopancreatography (ERCP) pancreatitis (PEP) is the most prevalent complication after ERCP with an incidence of 3.5%. PEP severity is classified according to either the consensus criteria or the revised Atlanta criteria. In this international cohort study we investigated which classification is the strongest predictor of PEP-related mortality. Methods: We reviewed 13,384 consecutive ERCPs performed between 2012 and 2017 in eight hospitals. We gathered data on all pancreatitis-related adverse events and compared the predictive capabilities of both classifications. Furthermore, we investigated the correlation between the two classifications and identified reasons underlying length of stay. Results: The total sample consisted of 387 patients. The revised Atlanta criteria have a higher sensitivity (100 vs. 55%), specificity (98 vs. 72%) and positive predictive value (58 vs. 5%). There is a significant difference (p < 0.001) between the two classifications. In 124 patients (32%), the length of stay was influenced by concomitant diseases. Conclusion: The revised Atlanta classification is superior in predicting mortality and better reflects PEP severity. This has important implications for researchers, clinicians and patients. For the diagnosis of PEP pancreatitis, the consensus criteria remain the golden standard. However, the revised Atlanta criteria are preferable for defining PEP severity. Key summary Established knowledge. Post-ERCP pancreatitis (PEP) is the most prevalent complication of ERCP, with an incidence rate of 3.5%. /home/ueg . Adequate assessment of PEP severity is necessary for researchers and clinicians to predict prognosis and compare efficacy of prophylactic measures for PEP. . Two classification systems for PEP severity exist: the consensus criteria and the revised Atlanta criteria. New findings. Our study shows that the diagnostic performance of the revised Atlanta criteria for PEP-related mortality is better than the consensus criteria, because of its focus on necrosis and organ failure. . In that way, use of the revised Atlanta criteria allows for a better and more objective evaluation of PEP prophylaxis efficacy. . However, the consensus criteria, with its focus on length of hospital stay, could still be useful in light of patient-reported outcome measures and patient-centered care.