BACKGROUND: Suspicion and clinical criteria continue to serve as the foundation for ventilator-associated pneumonia (VAP) diagnosis, however the criteria used to diagnose VAP vary widely. Data from head-to-head comparisons of clinical diagnostic algorithms is lacking, thus a prospective observational study was performed to determine the performance characteristics of the Johanson criteria, Clinical Pulmonary Infection Score (CPIS), and Centers for Disease Control and Prevention’s National Healthcare Safety Network (CDC/NHSN) criteria as compared to Hospital in Europe Link for Infection Control through Surveillance (HELICS) reference standard. METHODS: A prospective observational cohort study was performed in three mixed medical-surgical ICUs from one academic medical center from 1 October 2016 to 30 April 2018. VAP diagnostic criteria were applied to each patient including CDC/NHSN, CPIS, HELICS and Johanson criteria. Tracheal cultures and serum procalcitonin values were obtained for each patient. RESULTS: Eighty-five patients were enrolled (VAP 45, controls 41), mean age 46.94±18.9 years with a male predominance (72.94%). Using HELICS as the reference standard, the true positive (TP; sensitivity) and false negative (FN; miss rate) rates were CDC/NHSN (TP 44%; FN 0%), CPIS (TP 43%; FN 1%), Johnson (TP 43%; FN 1%). The highest true negative rate was seen with CPIS. CPIS had the highest Youden index; CDC/NHSN had the lowest. The positive tracheal culture rate was 81.2%. The sensitivity for positive tracheal culture with the serum procalcitonin level >0.5 ng/ml was 51.8%. CDC/NHSN had the highest false positive correlation with tracheal aspirate cultures. CONCLUSION: VAP remains a considerable source of morbidity and mortality in modern ICUs. The optimal diagnostic method remains unclear. Using HELICS criteria as the reference standard, CPIS displayed greater diagnostic accuracy compared to CDC/NHSN and Johanson criteria. Accuracy was improved with the addition of serum procalcitonin >0.5 ng/ml, but not positive quantitative endotracheal aspirate culture.TRIAL REGISTRATION: Not indicated for this study type.