Cancer-associated fibroblasts (CAFs) play a key role in cancer progression by contributing to extracellular matrix (ECM) deposition and remodeling, extensive crosstalk with cancer cells, epithelial-to-mesenchymal transition (EMT), invasion, metastasis, and therapy resistance. As metastasis is a main reason for cancer-related deaths, it is crucial to understand the role of CAFs in this process. Colorectal cancer (CRC) is a heterogeneous disease and lethality is especially common in a subtype of CRC with high stromal infiltration. A key component of stroma is cancer-associated fibroblasts (CAFs). To provide new perspectives for research on CAFs and CAF-targeted therapeutics, especially in CRC, we discuss the mechanisms, crosstalk, and functions involved in CAF-mediated cancer invasion, metastasis, and protection. This summary can serve as a framework for future studies elucidating these roles of CAFs.