Despite the considerable research attention paid to block copolymer (BCP)-toughened epoxy resins, the effects of their terminal groups on their phase structure are not thoroughly understood. This study fills this gap by closely examining the effects of amino and carboxyl groups on the fracture toughness of epoxy resins at different temperatures. Through the combination of scanning electron microscopy and digital image correlation (DIC), it was found that the amino-terminated BCP was capable of forming a stress-distributing network in pure epoxy resin, resulting in better toughening effects at room temperature. In a 60 wt.% silica-filled epoxy composite system, the addition of a carboxyl-terminated BCP showed little toughening effect due to the weaker filler/matrix interface caused by the random dispersion of the microphase of BCPs and distributed silica. The fracture toughness of the epoxy system at high temperatures was not affected by the terminal groups, regardless of the addition of silica. Their dynamic mechanical properties and thermal expansion coefficients are also reported in this article.