In order to achieve an efficient ballistic protection at a low weight, it is necessary to deeply explore the energy absorption mechanisms of ballistic fabric structures. In this paper, finite element (FE) yarn-level models of the designed three-dimensional (3D) angle-interlock (AI) woven fabrics and the laminated two-dimensional (2D) plain fabrics are established. The ballistic impact responses of fabric panels with and without the interlocking Z-warp yarns during the projectile penetration are evaluated in terms of their energy absorption, deformation, and stress distribution. The Z-warps in the 3D fabrics bind different layers of wefts together and provide the panel with structural support along through-the-thickness direction. The results show that the specific energy absorption (SEA) of 3D fabrics is up to 88.1% higher than that of the 2D fabrics. The 3D fabrics has a wider range of in-plane stress dispersion, which demonstrates its structural advantages in dispersing impact stress and getting more secondary yarns involved in energy absorption. However, there is a serious local stress concentration in 2D plain woven fabrics near the impact location. The absence of Z-warps between the layers of 2D laminated fabrics leads to a premature layer by layer failure. The findings are indicative for the future design of ballistic amors.