The dietary composition has been approved to be strongly associated with the risk of colorectal cancer (CRC), one of the most serious malignancies worldwide, through regulating the gut microbiota structure, thereby influencing the homeostasis of colonic epithelial cells by producing carcinogens, i.e., ammonia or antitumor metabolites, like butyrate. Though butyrate-producing Fusobacterium nucleatum has been considered a potential tumor driver associated with chemotherapy resistance and poor prognosis in CRC, it was more frequently identified in the gut microbiota of healthy individuals rather than CRC tumor tissues. First, within the concentration range tested, the fermentation broth of F. nucleatum exhibited no significant effects on Caco-2 and NCM460 cells viability except for a notable up-regulation of the expression of TLR4 (30.70%, p < 0.0001) and Myc (47.67%, p = 0.021) and genes encoding proinflammatory cytokines including IL1B (197.57%, p < 0.0001), IL6 (1704.51%, p < 0.0001), and IL8 (897.05%, p < 0.0001) in Caco-2 cells exclusively. Although no marked effects of polydextrose or fibersol-2 on the growth of F. nucleatum, Caco-2 and NCM460 cells were observed, once culture media supplemented with polydextrose or fibersol-2, the corresponding fermentation broths of F. nucleatum significantly inhibited the growth of Caco-2 cells up to 48.90% (p = 0.0003, 72 h, 10%) and 52.96% (p = 0.0002, 72 h, 10%), respectively in a dose-dependent manner. These two kinds of fibers considerably promoted butyrate production of F. nucleatum up to 205.67% (p < 0.0001, 6% polydextrose at 24 h) and 153.46% (p = 0.0002, 6% fibersol-2 at 12 h), which explained why and how the fermentation broths of F. nucleatum cultured with fibers suppressing the growth of Caco-2 cells. Above findings indicated that dietary fiber determined F. nucleatum to be a carcinogenic or antitumor bacterium, and F. nucleatum played an important role in the association between the dietary composition, primarily the content of dietary fibers, and the risk of CRC.