A significant proportion of babies that are admitted to the neonatal intensive care unit (NICU) suffer from late onset sepsis (LOS). In order to prevent mortality and morbidity, the early detection of LOS is of the utmost importance. Recent works have found that the use of machine learning techniques might help detect LOS at an early stage. Some works have shown that linear methods (i.e., logistic regression) display a superior performance when predicting LOS. Nevertheless, as research on this topic is still in an early phase, it has not been ruled out that non-linear machine learning (ML) techniques can improve the predictive performance. Moreover, few studies have assessed the effect of parameters other than heart rate variability (HRV). Therefore, the current study investigates the effect of non-linear methods and assesses whether other vital parameters such as respiratory rate, perfusion index, and oxygen saturation could be of added value when predicting LOS. In contrast with the findings in the literature, it was found that non-linear methods showed a superior performance compared with linear models. In particular, it was found that random forest performed best (AUROC: 0.973), 24% better than logistic regression (AUROC: 0.782). Nevertheless, logistic regression was found to perform similarly to some non-linear models when trained with a short training window. Furthermore, when also taking training time into account, K-Nearest Neighbors was found to be the most beneficial (AUROC: 0.950). In line with the literature, we found that training the models on HRV features yielded the best results. Lastly, the results revealed that non-linear methods demonstrated a superior performance compared with linear methods when adding respiratory features to the HRV feature set, which ensured the greatest improvement in terms of AUROC score.