Our previous research has demonstrated that colorectal cancer (CRC) progression was promoted by circN4BP2L2. This study aimed to further explore the mechanism of circN4BP2L2 in the development of CRC from the perspective of small extracellular vesicles (sEVs). Cancer-associated fibroblasts cell (CAFs) and normal fibroblasts cell (NFs) were isolated from CRC tissues and adjacent tissues, respectively. The ultra-centrifugation was used for extraction of their related sEVs. Cell proliferation and apoptosis were analyzed using CCK-8 and flow cytometry, respectively. Transwell assay was conducted to measure cell migration. The tube formation ability was assessed by tube formation assay. The target relationships between circN4BP2L2 and miR-664b-3p, and miR-664b-3p and HMGB3 were validated by dual-luciferase reporter detection. The effect of CAFs-derived sEV (CAFs-sEVs) circN4BP2L2 on CRC was further studied in nude mice. CAFs-exo promoted cell proliferation, migration, tube formation ability, and inhibited apoptosis of CRC cells. CAFs-sEV circN4BP2L2 knockdown reversed the above results. CircN4BP2L2 directly targeted miR-664b-3p, and HMGB3 was targeted by miR-664b-3p. Moreover, subcutaneous tumorigenesis and liver metastasis of nude mice with CRC were repressed by CAFs-sEV circN4BP2L2 knockdown. CAFs-sEV circN4BP2L2 knockdown restrained CRC cell proliferation and migration by regulating miR-664b-3p/HMGB3 pathway.