In this paper, the effect of addition of nanosilica on mechanical properties of pure epoxy and epoxy/fiberglass composite has been investigated. The epoxy/nanosilica composites and epoxy/fiberglass/nanosilica hybrid composites have been fabricated, and the Young’s modulus, tensile strength, yield stress and elongation at break have been determined by simple extension tests. The results show that by addition of 1 wt% of nanosilica in both types of composites, Young’s modulus, yield stress and tensile strength decrease and elongation at break increases. By increasing the nanosilica content, the Young’s modulus, yield stress and tensile strength increase and elongation at break decreases. Also, imperialist competitive algorithm is employed to model the mechanical properties as fourth degree polynomial functions. The accuracy of polynomial is maximized and coefficients are obtained. The results show 25.66%, 56.87% and 45.84% improvement in Young’s modulus, yield stress and tensile strength of pure epoxy, respectively. Also, 12.9%, 24.83% and 12.85% improvement in Young’s modulus, yield stress and tensile strength of epoxy/fiberglass composites, has been observed, respectively.