D-A conjugated polymers are key materials for organic solar cells and organic thin-film transistors, and their film structure is one of the most important factors in determining device performance. The formation of film structure largely depends on the crystallization process, but the crystallization of D-A conjugated polymers is not well understood. In this review, we attempted to achieve a clearer understanding of the crystallization of D-A conjugated polymers. We first summarized the features of D-A conjugated polymers, which can affect their crystallization process. Then, the crystallization process of D-A conjugated polymers was discussed, including the possible chain conformations in the solution as well as the nucleation and growth processes. After that, the crystal structure of D-A conjugated polymers, including the molecular orientation and polymorphism, was reviewed. We proposed that the nucleation process and the orientation of the nuclei on the substrate are critical for the crystal structure. Finally, we summarized the possible crystal morphologies of D-A conjugated polymers and explained their formation process in terms of nucleation and growth processes. This review provides fundamental knowledge on how to manipulate the crystallization process of D-A conjugated polymers to regulate their film structure.