Cured unsaturated polyester resins are often highly brittle. In this paper, to improve fatigue life of the glass fiber‐reinforced polyester composite (GFRP) used in the wind turbine blade, the unsaturated polyester resin was toughened by 2 wt% liquid carboxyl‐terminated butadiene acrylonitrile (CTBN) copolymer. CTBN liquid rubber has acidic OH of the carboxylic group, which causes incompatibility and poor solubility of CTBN in unsaturated polyester resins. In this study, to increase compatibility and solubility of CTBN liquid rubber in unsaturated polyester resins, 1,4‐Butanediol diglycidyl ether (1,4‐BDGE) was selected to react with carboxylic group of the CTBN liquid rubber which converted to ester functional group. The reactions between epoxy groups of 1,4‐BDGE and carboxyl groups of CTBN were investigated by Fourier‐transform infrared (FTIR) spectroscopy. Four polyester samples were prepared by mixing unsaturated polyester with 0 to 3 wt% of modified CTBN. Tensile tests of polymer samples have been performed to observe the effect of modified CTBN additive. According to the results of the tensile test, the highest tensile strength, elongation‐at‐break, and modulus of toughness were observed in polyester with 2 wt% rubber. Two polyester/glass fiber composite samples with and without 2 wt% rubber were prepared by the vacuum infusion process (VIP) and subjected to fatigue behavior analysis. According to the results of the static tension test, the same value of tensile strength was observed in both composites. The obtained tension‐tension fatigue results (R = 0.1) showed a reduction in maximum stress level enhanced the fatigue life improvement for the samples with 2 wt% rubber due to the increased toughness of the composite matrix. Examining the fracture surfaces of the specimens after fatigue tests, it was revealed that failure of the polyester containing 2 wt% rubber/glass fiber occurred at the interface, while the specimens without rubber showed that failure did not occur at the fiber/matrix interface.