Incorporation of short wood fillers such as wood flour (WF) into polypropylene (PP) often results in a marked reduction of toughness, which is one of the main shortcomings for WF/PP composites. This research reports a facile approach to achieve toughening of WF/PP composites via introducing self-assembling β-nucleating agents into PP matrix. The effect of two kinds of nucleating agents, an aryl amide derivative (TMB5) and a rare earth complex (WBG II), at varying concentrations on the crystallization and mechanical properties of WF/PP composites was comparatively investigated. The results showed that both nucleating agents were highly effective in inducing β-crystal for WF/PP, with β-crystal content (kβ) value reaching 0.8 at 0.05 wt% nucleating agent concentration. The incorporation of TMB or WBG significantly decreased the spherulite size, increased the crystallization temperature and accelerated the crystallization process of WF/PP. As a result of PP crystalline modification, the toughness of composites was significantly improved. Through introducing 0.3 wt% TMB or WBG, the notched impact strength and strain at break of WF/PP increased by approximately 28% and 40%, respectively. Comparatively, although WF/PP-WBG had slightly higher Kβ value than WF/PP-TMB at the same concentration, WF/PP/TMB exhibited more uniform crystalline morphology with smaller spherulites. Furthermore, the tensile strength and modulus of WF/PP-TMB were higher than WF/PP-WBG. This matrix crystalline modification strategy provides a promising route to prepare wood filler/thermoplastic composites with improved toughness and accelerated crystallization.