Based on the conjugate gradient method, we propose a modified iterative method (MCG) for impulse noise removal based on the famous two-phase scheme. The noise candidates are firstly identified by the adaptive median filter, and then the MCG method recovers these noise candidates by minimizing an edge-preserving regularization function. A nice property of the MCG method is that the global convergence can be established without assuming that the objective function satisfies strong convexity under the Armijo-type line search. Numerical experiments show that the MCG method is effective to remove the impulse noise.