In this paper, both location and energy of impacts on an anisotropic carbon fiber reinforced plate (CFRP) are detected with the help of deep learning. We introduce sparse low-cost sensor array integration in CFRP plates that allows for structural monitoring of lightweight structures. Using a resin transfer moulding process microelectromechanical systems (MEMS) and piezoelectric transducers (PZT) sensors are integrated into CFRP plates. We developed an automated test bench to perform weight drop impact loadings with impact energies ranging between 0.22–0.56 mJ on a 1 × 1 cm2-grid with 441 locations. The obtained sensor signals were processed by means of a short-time fourier transformation and used as input for the training of a deep learning model. This model was implemented with a convolutional neural network. To accelerate the training phase we introduce a coarse analytical model that generates artificial sensor signals we use for pretraining of the neural network. Yielding high prediction accuracies of 99.82% and 98.68% for a correct classification of impact location and energy, respectively, the capability of the proposed approach was demonstrated. Despite their limited resolution the low-cost MEMS accelerometers were able to correctly locate an impact and its energy with 99.76% and 97.04%, respectively. The pretraining led to an increased robustness of the training process. Additionally, for the case of PZT sensors, it also reduced the number of required epochs for convergence significantly.