Tumor budding scoring guidelines from the International Tumor Budding Consensus Conference (ITBCC) for colorectal cancer propose three groups: BD1 (0–4 buds/0.785 mm2), BD2 (5–9 buds/0.785 mm2), and BD3 (10 or more buds/0.785 mm2). Here, we investigate whether a fourth scoring category, namely zero buds, may have additional clinical relevance. The number of tumor buds/0.785 mm2 was scored in 959 cases. Those with zero tumor buds were considered BD0, while a new BD1 category of 1–4 buds was proposed. Associations of both scoring approaches with clinicopathological features were analyzed. Conventional ITBCC scoring showed expected associations with unfavorable histopathological prognostic factors. In total, 111/959 (11.6%) were BD0. A significant difference was found when BD0 was compared statistically to BD1 (1–4 buds) for pT, TNM, tumor grade, and lymphatic, venous, and perineural invasion (p < 0.01, all). Tumors with BD0 occur relatively frequently and contribute additional information on tumor behavior. BD0 should be considered for subsequent ITBCC guidelines.