Total variation (TV) regularization can very well remove noise and simultaneously preserve the sharp edges. But it has the drawback of the contrast loss in the restoration. In this paper, we first theoretically analyze the loss of contrast in the original TV regularization model, and then propose a forward-backward diffusion model in the framework of total variation, which can effectively preserve the edges and contrast in TV image denoising. A backward diffusion term based on a nonconvex and monotony decrease potential function is introduced in the TV energy, resulting in a forward-backward diffusion. In order to finely control the strength of the forward and backward diffusion, and separately design the efficient algorithm to numerically implement the forward and backward diffusion, we propose a two-step splitting method to iteratively solve the proposed model. We adopt the efficient projection algorithm in the dual framework to solve the forward diffusion in the first step, and then use the simple finite differences scheme to solve the backward diffusion to compensate the loss of contrast occurred in the previous step. At last, we test the models on both synthetic and real images. Compared with the classical TV, forward and backward diffusion (FBD), two-step methods (TSM), and TV-FF models, our model has the better performance in terms of peak signal-to-noise ratio (PSNR) and mean structural similarity (MSSIM) indexes.