The Kriging-based reliability method with a sequential design of experiments (DoE) has been developed in recent years for implicit limit state functions. Such methods include the efficient global reliability analysis, the active learning reliability method combining Kriging and MCS Simulations. In this research, a novel local approximation method based on the most probable failure point (MPFP) is proposed to improve such methods. In this method, the MPFP calculated in the last iteration is the center of the next sampling region. The size of the local region depends on the reliability index obtained by the First Order Reliability Method (FORM) and the deviation distance of the standard deviation. The proposed algorithm, which approximates the limit state function accurately near MPFP rather than in the whole design space, can avoid selecting samples in regions that have negligible effects on the reliability analysis results. In addition, a multi-point enrichment technique is also introduced to select multiple sample points in each iteration. After the high-quality approximation of limit state function is obtained, the failure probability is calculated by the Monte Carlo method. Four numerical examples are used to validate the accuracy and efficiency of the proposed method. Results show that the proposed method is very effective for an accurate evaluation of the failure probability.