An ultrasonic coupling agent, as an acoustic medium between the ultrasonic probe and the surface of the specimens, is indispensable in Nondestructive Testing (NDT). Whether it is liquid, air, or solid coupling agent, the problem of improving the efficiency of ultrasonic propagation in a coupling agent is one worth studying. Glycerol and hydrogels are two common liquid coupling agents in NDT. This study intended to investigate the effect of graphene addition on the performance of these coupling agents in NDT. Firstly, based on the theory of acoustic impedance matching, the authors established an index system to evaluate the performance of ultrasonic coupling agent by experiments. Secondly, hydrogel–graphene and glycerol–graphene composite coupling agents were prepared by adding three-dimensional graphene structure powders with mass fraction of 0.25%, 0.5%, 0.75%, and 1% to CG-98 hydrogel coupling agent and HG-99 glycerol coupling agent, respectively. Corresponding experiments were conducted on these composite coupling agents. Peak-to-peak value, attenuation coefficient, and energy value of first echo are calculated at different frequencies. The experimental results showed that graphene can significantly improve the ultrasonic propagation performance of hydrogel and glycerin coupling agents. In addition, when the mass fraction of graphene added was 0.75%, the coupling agent had the best performance. Finally, we measured the acoustic impedance values of the composite couplings with different graphene contents to demonstrate the reliability of the experimental results.