The present paper aims to compare the predictive performance of five models namely the Linear Discriminant Analysis (LDA), Logistic Regression (LR), Decision Trees (DT), Support Vector Machine (SVM) and Random Forest (RF) to forecast the bankruptcy of Tunisian companies. A Deep Neural Network (DNN) model is also applied to conduct a prediction performance comparison with other statistical and machine learning algorithms. The data used for this empirical investigation covers 25 financial ratios for a large sample of 732 Tunisian companies from 2011–2017. To interpret the prediction results, three performance measures have been employed; the accuracy percentage, the F1 score, and the Area Under Curve (AUC). In conclusion, DNN shows higher accuracy in predicting bankruptcy compared to other conventional models, whereas the random forest performs better than other machine learning and statistical methods.