In the environment of electronic warfare, the countermeasure between radar and jammer has become a hot issue. Finding the solution to optimal waveforms between them is very great significance. Aiming at the problem of multi-target detection in clutter environment, the authors propose the waveform optimization method based on signal-to-interference-plus-noise ratio (SINR) criterion for radar and jammer, respectively. When radar is the dominant player, the maxmin strategy is used to establish the optimization model to optimize the radar transmission waveform. When jammer is the dominant player, the minmax strategy is used to establish the optimization model to optimize the jammer waveform. When dealing with random multi-target, the authors also propose an adaptive weight method to solve the problem of summation of weighted multiple targets. Then, an improved method combined with Jensen’s inequality for radar and jammer is proposed, respectively, which could improve the detection performance of the radar system and reduce the computational complexity of maximum SINR-based method. The simulation results show that the proposed maximum SINR-based methods could adaptively allocate the transmission energy to improve the performance of the radar. The energy allocation of radar and jammer is slightly different under maxmin strategy and minmax strategy. The proposed improved methods could output the higher SINR than the maximum SINR-based methods. Both of the proposed methods could improve the performance of radar detecting targets.