An optimized “rigid-flexible” structure with multistage gradient modulus was constructed on carbon fiber (CF) surface via chemical grafting using “flexible” polyethyleneimine (PEI) and “rigid” polydopamine (PDA) between “rigid” CF and “flexible” epoxy (EP) to elaborate a double alternant “rigid-flexible” structure for simultaneously strengthening and toughening CF/EP composites. PDA and PEI polymers can greatly enhance the roughness and wettability of CF surfaces, further strengthening the mechanical interlocking and chemical interactions between CFs and epoxy. Besides, the “rigid-flexible” structure endows the interface with a gradient transition modulus, which could uniformly transfer internal stress and effectively avoid the stress concentration. Moreover, the double alternant “rigid-flexible” could buffer the external loading, induce more micro cracks and propagation paths and, thereby, consume more energy during the destruction of the composite. The interfacial shear strength, interlaminar shear strength, impact strength increased by 80.2%, 23.5% and 167.2%, and the fracture toughness improved by 227.2%, compared with those of the unmodified CF composite, respectively. This creative strategy and design afford a promising guidance for the preparation and production of advanced CF/EP structural materials with high strength and toughness.