Hospital intensive care units (ICUs) care for severely ill patients, many of whom require some form of organ support. Clinicians in ICUs are often challenged with integrating large volumes of continuously recorded physiological and clinical data in order to diagnose and treat patients. In this work, we focus on developing interpretable models for predicting unexpected respiratory decompensation requiring intubation in ICU patients. Predicting need for intubation could have important implications for the patient and medical staff and potentially enable timely interventions for improved patient outcome. Using data from adult ICU patients from the Medical Information Mart for Intensive Care (MIMIC)-III database, we developed gradient boosting models for predicting intubation onset. In a cohort of 12,470 patients, of whom 1,067 were intubated (8.55%), we achieved an area under the receiver operating characteristic curve (AUROC) of 0.89, with 95% confidence interval (CI) 0.87-0.91, when predicting intubation 3 hours ahead of time, a significant increase (p<0.001) over the AUROC achieved using several baselines, including logistic regression (0.81, 95% CI 0.78-0.84) and neural networks (0.80, 95% CI 0.77-0.83]). Finally, we conducted feature importance analysis using gradient boosting and derived useful insights in understanding the relative importance of clinical vs. biological variables in predicting impending respiratory decompensation in ICUs.