Software reliability always related to software failures, in a past few decades a software reliability growth models (SRGMs) number have been developed to predict the software reliability under different environment, but there is no single model that best fits all the real life situations and so can be recommended universally. to predict the failures of software accurately, an appropriate and best model must be chosen, this will help to estimate the cost and delivery time of the project. In this paper, Ant Lion optimization (ALO) algorithm is proposed to optimize estimation of parameters and a choice procedure is used to select an appropriate model of the software reliability that best fit available dataset of an ongoing project of the software. Employing ALO algorithm for estimating the SRGM's parameters has provided more accurate prediction and enhance procedure of the selection, making a decision to select suitable SRGMs during the phases of the testing can be more easier to a developer of the software .The explored algorithm has been examined on various datasets of software projects and it has been noticed that this method is better than other methods proposed.