To improve the tribological properties of bismaleimide (BMI) resin, silicon dioxide nanoparticles with imino and terminal functional amino groups were prepared through a sol–gel process to form a novel SiO2-NH2 hybrid. The as-prepared hybrid was then applied as a modifying agent for the BMI matrix to obtain SiO2-NH2/BMI composites. Compared to those of pure BMI resin, the volume wear rate and friction coefficient of the SiO2-NH2/BMI composites decreased significantly, while the wear mechanism changed from fatigue (BMI) to adhesive (SiO2-NH2/BMI) wear. This improvement in the tribological properties of the SiO2-NH2/BMI composites was attributed to the appropriate SiO2-NH2 added content, which endowed the BMI with excellent mechanical and thermal-resistant properties. Thus, the SiO2-NH2/BMI composites could resist the external load and excessive heat during the friction process.