The partial aminolysis of poly(styrene-alt-maleic anhydride) (SMA) with 3-aminopropyltriethoxysilane (APTES) was carried out followed by in situ hydrolytic condensation of the silane, for inducing the self-assembly of SMA to prepare an organic-inorganic hybrid colloidal particle (SMA-APTES). The obtained SMA-APTES particles were then employed as a toughener for modifying diglycidylether of bisphenol A/anhydride thermoset. When the additive amount of the modifier was 5 wt %, an optimal overall performance was achieved. Compared to neat epoxy thermoset, the tensile strength, elongation-at-break, impact strength, and fracture toughness were increased by 67.6, 159, 170.5, and 230.9%, respectively. Although SMA-APTES particles inherently had low glass-transition temperature (T g ) and thermal stability, the T g and thermal stability of above case were comparable to that of neat epoxy resin. It was contributed by the remarkable increase in the crosslink density offsetting this shortage. Therefore, SMA-APTES particle was proved to be an effective and promising modifier for epoxy thermoset.