Hybrid bonded–bolted composite material interference connections significantly enhance the collaborative load-bearing capabilities of the adhesive layer and bolts, thus improving structural load-carrying capacity and fatigue life. So, these connections offer significant developmental potential and application prospects in aircraft structural assembly. However, interference causes damage to the adhesive layer and composite laminate around the holes, leading to issues with interface damage. In this study, we employed experimental and finite element methods. Initially, different interference-fit sizes were selected for bolt insertion to analyze the damage mechanism of the adhesive layer during interference-fit bolt installation. Subsequently, a finite element tensile model considering damage to the adhesive layer and composite laminate around the holes post-insertion was established. This study aimed to investigate damage in composite bonded–bolted hybrid joints, explore load-carrying rules and failure modes, and reveal the mechanisms of interference effects on structural damage and failure. The research results indicate that the finite element prediction model considering initial damage around the holes is more effective. As the interference-fit size increases, damage to the adhesive layer transitions from surface debonding to local cracking, while damage to the composite matrix shifts from slight compression failure to severe delamination and fiber-bending fracturing. The structural strength shows a trend of initially increasing and then decreasing, with the maximum strength observed at an interference-fit size of 1.1%.