To tackle the problem of partial loss of image details in infrared and visible image fusion, a gradient transfer optimization model is proposed for the fusion of infrared and visible images. Firstly, an adaptive image decomposition method is proposed based on coupled partial differential equation, the infrared image and the visible image are decomposed into base layer and detail layer to extract the highbrightness target and the details of the two images. Based on this superior information of infrared image and visible image, the optimization model is designed to obtain the fusion image obvious target and rich details. For the proposed optimization model, Alternating Direction Method of Multipliers (ADMM) is used to decompose the original model into sub-problems that are easy to solve and iteratively optimize to obtain the optimal solution. The introduction of control parameters makes the model more flexible in different situations, and retains the thermal radiation information of the infrared image and the detailed information of the visible image to the greatest extent. The fused image visual effects and performance indicators are improved. We completed the experiment using a public data set and analyzed the experimental results. The experimental results show that the proposed method can better preserve the clear target and texture information of infrared and visible images, and the fusion results are more accurate and comprehensive. The experiment results also indicate that our method performs well and achieves comparable metric values with the state-of-the-art methods. INDEX TERMS Image fusion, optimization model, ADMM, partial differential equation.