Polylactide is a highly demanded biopolymer, whose industrial application constantly increases. Its disadvantages such as brittleness and slow crystallization rate can be overcome by application of different additives. Because of environmental issues, using natural waste fillers as modifying agents for polylactide is especially interesting. In this study linseed cake, a byproduct of oil extraction from linseed, characterized by oil content of 0.9–39.8 wt.%, was added to polylactide to influence its crystallization behavior. The formation of the crystalline phase was studied by differential scanning calorimetry in isothermal and non-isothermal conditions and analyzed according to methods by Jeziorny, Ozawa, Mo and Avrami. The samples’ microstructures were observed using polarized light microscopy. The crystallization rate and Avrami exponent of samples crystallized in different conditions were evaluated. It was found that addition of 10 wt.% of linseed cake containing at least 17.7 wt.% oil notably changes the crystallization of polylactide, increasing its crystallinity and promoting the growth of crystallites.