There has been an increase in the use of ultrasonic arrays for the detection of defects in composite structures used in the aerospace industry. The response of a defect embedded in such a medium is influenced by the inherent anisotropy of the bounding medium and the layering of the bounding medium and hence poses challenges for the interpretation of the full matrix capture (FMC) results. Modeling techniques can be used to understand and simulate the effect of the structure and the defect on the received signals. Existing modeling techniques, such as finite element methods (FEM), finite difference time domain (FDTD), and analytical solutions, are computationally inefficient or are singularly used for structures with complex geometries. In this paper, we develop a novel model based on the Gaussian-based recursive stiffness matrix approach to model the scattering from a side-drilled hole embedded in an anisotropic layered medium. The paper provides a novel method to calculate the transmission and reflection coefficients of plane waves traveling from a layered anisotropic medium into a semi-infinite anisotropic medium by combining the transfer matrix and stiffness matrix methods. The novelty of the paper is the developed model using Gaussian beams to simulate the scattering from a Side Drilled Hole (SDH) embedded in a multilayered composite laminate, which can be used in both immersion and contact setups. We describe a method to combine the scattering from defects with the model to simulate the response of a layered structure and to simulate the full matrix capture (FMC) signals that are received from an SDH embedded in a layered medium. The model-assisted correction total focusing method (MAC-TFM) imaging is used to image both the simulated and experimental results. The proposed method has been validated for both isotropic and anisotropic media by a qualitative and quantitative comparison with experimentally determined signals. The method proposed in this paper is modular, computationally inexpensive, and is in good agreement with experimentally determined signals, and it enables us to understand the effects of various parameters on the scattering of a defect embedded in a layered anisotropic medium.