Based on the cohesive zone model, a meso-scale model is developed for numerical studies of three-phase concrete under tension and compression. The model is characterized by adopting mixed-mode fracture and interaction behavior to describe fracture, friction and collision in tension and compression processes. The simulation results match satisfactorily with the experimental results in both mechanical characteristics and failure mode. Whole deformation and crack propagation process analyses are conducted to reveal damage evolution of concrete. The analyses also set a foundation for the following parametric studies in which mode II fracture energy, material parameter, frictional angle and aggregates’ mechanical characteristics are considered as variables. It shows that the mixed-mode fracture accounts for a considerable proportion, even in tension failure. Under compression, the frictional stress can constrain crack propagation at the beginning of the damage and reestablish loading path during the softening stage. Aggregates’ mechanical characteristics mainly affect concrete’s performance in the mid-and-late softening stage.