Analytical performance evaluation of a digital communication system remains a serious problem especially when a sophisticated digital signal processing is considered. Moreover, it is difficult to obtain the expected performance of such system using the Monte Carlo simulation method. In this paper, we propose a new semi-analytical approach for predicting error probability in a digital communication system. This approach is based on Fourier transform inversion formula to estimate the probability density function (pdf) of the observed soft sample at the receiver. Furthermore, we applied a bootstrap method for selecting the optimal smoothing parameter to make the proposed semi-analytical method more accurate. Simulation results show that the obtained semi-analytical error probability is close to the one measured using Monte Carlo simulation and provides a significant gain in terms of computing time. Besides, the use of the bootstrap method decreases the squared error between the true pdf and the estimated one.