A comparative experimental and numerical study of the impact behaviour of carbon-fiber-reinforced thermoplastic (TP) and thermoset (TS) composites has been carried out. On the one hand, low velocity impact (LVI) tests were performed on TP and TS composites with different lay-up sequences at different energy levels, and the damage modes and microscopic damage mechanisms after impact were investigated using macroscale inspection, C-scan inspection, and X-ray-computed tomography. The comparative results show that the initial damage valve force under LVI depends not only on the material, but also on the layup sequence. The initial valve force of the P2 soft layer with lower stiffness is about 11% lower than that of the P1 quasi-isotropic layer under the same material, while the initial valve force of thermoplastic composites is about 28% lower than that of thermoset composites under the same stacking order. Under the same stacking order and impact energy level, the damage area and depth of TP composites are smaller than those of TS composites; while under the same material and impact energy level, the indentation depth of P2 plies is greater than that of P1 plies, and the damage area of P2 plies is smaller than that of P1 plies, but the change of thermoplastic composites is not as obvious as that of thermoset composites. This indicates that TP composites have a higher initial damage threshold energy and impact resistance at the same lay-up order, while increasing the lay-up ratio of the same material by 45° improves the impact resistance of the structure. In addition, a damage model based on continuum damage mechanics (CDM) was developed to predict different damage modes of thermoplastic composites during low velocity impact, and the analytical results were compared with the experimental results. At an impact energy of 4.45 J/mm, the error of the initial damage valve force is 5.26% and the error of the maximum impact force is 4.36%. The simulated impact energy and impact velocity curves agree with the experimental results, indicating that the finite element model has good reliability.