Inorganic fillers of whisker-like morphology are considered promising materials, and they have gained significant attention in recent years as a substitute for glass fibers due to their fibrous surface characteristics and extremely low bulk density. This study selected magnesium sulfate whiskers (MSWs) containing crystal water and pure calcium sulfate whiskers (CSWs) as fillers. They were physically blended with biodegradable polymers PBAT and PBS in a range of 0.1 wt.% to 2 wt.% to form composite materials. The non-isothermal crystallization behavior of the composite materials was studied using differential scanning calorimetry (DSC). The data indicated that with an increase in whisker content, the crystallization temperature (Tc) of composites increased, and the addition of a small amount of whiskers led to a reduction in the half-crystallization time of the composite materials, indicating the crystallization capacity of the materials has been enhanced to varying degrees. Analysis via POM unveiled a consistent pattern of decreased spherulite size and heightened spherulite count with the introduction of whiskers. This phenomenon is ascribed to the whisker fillers’ function as nucleating agents within the polymer matrices, thereby stimulating the crystallization process. Interestingly, the findings suggest that CSWs exert a more significant influence on crystallization than MSWs, likely due to the distinct single fiber morphology of the former filler.