Designing composite materials with tailored stiffness and toughness is challenging due to the massive number of possible material and geometry combinations. Although various studies have applied machine learning techniques and optimization methods to tackle this problem, we still lack a complete understanding of the material effects at different positions and a systematic experimental procedure to validate the results. Here we study a two-dimensional (2D) binary composite system with an edge crack and grid-like structure using a Genetic Algorithm (GA) and Conditional Variational Autoencoder (CVAE), which can design a composite with desired stiffness and toughness. The fitness of each design is evaluated using the negative mean square error of their predicted stiffness and toughness and the target values. We use finite element simulations to generate a machine-learning dataset and perform tensile tests on 3D-printed specimens to validate our results. We show that adding soft material behind the crack tip instead of ahead of the tip tremendously increases the overall toughness of the composite. We also show that while GA generates composite designs with slightly better accuracy (both methods perform well, with errors below 20%), CVAE takes considerably less time (~1/7500) to generate designs. Our findings may provide insights into the effect of adding soft material at different locations of a composite system and may also provide guidelines for conducting experiments and Explainable Artificial Intelligence (XAI) to validate the results.