Superconducting coils made of Rare-Earth-Barium-Copper-Oxide (REBCO) coated conductor (CC) exhibit superior electromagnetic performance. Employing epoxy impregnation can improve the structural integrity and mechanical property of the superconducting coils. However, due to the extreme work environment and weak adhesion strength of REBCO CC, the delamination induced by radial thermal stress and electromagnetic force significantly affects the electromagnetic property and the reliability of the superconducting coil. This study proposes a three-dimensional (3D) thermal-electromagnetic mechanical delamination model that incorporates the cohesive zone model (CZM) to investigate the delamination mechanisms in epoxy impregnated REBCO pancake coils during the cooling and coil operation processes. The simulation employs a three-parameter Weibull distribution to account for the inhomogeneity of transverse tensile strength in the CCs. The delamination behavior and mechanisms of the coils under different conditions are analyzed. The simulation results show that the model considering random adhesion strength proves to be more effective in representing the delamination behavior of the coil. And large tensile radial stresses caused by thermal stresses and electromagnetic forces lead to the delamination behavior of the coil during cooling and operation. The main reason for the tensile radial stress is the mismatch in the thermal contraction among components of the coils during cooling process. Furthermore, we investigate the influence of the thermal expansion coefficient (CTE) and thickness of the mandrel, the CTE and prestress of the overband and the initial localized damage. The results indicate that these factors significantly affect the tensile radial stress and the extent of delamination in the windings. And the extent and distribution of delamination is related to the stress release caused by delamination to a certain degree.