The novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is the culprit behind the coronavirus disease 2019 (COVID-19), which has killed millions of people. SARS-CoV-2 binds its spike (S) protein to the angiotensin-converting enzyme 2 (ACE2) receptor to inter the epithelial cells in the respiratory tracts. ACE2 is a crucial mediator in the SARS-CoV-2 infection pathway. In this paper, we construct a mathematical model to describe the SARS-CoV-2/ACE2 interaction and the adaptive immunological response. The model predicts the effects of latently infected cells as well as immunological responses from cytotoxic T lymphocytes (CTLs) and antibodies. The model is incorporated with three distributed time delays: (i) delay in the formation of latently infected epithelial cells, (ii) delay in the activation of latently infected epithelial cells, (iii) delay in the maturation of new released SARS-CoV-2 virions. We show that the model is well-posed and it admits five equilibria. The stability and existence of the equilibria are precisely controlled by four threshold parameters Ri, i=0,1,2,3. By formulating suitable Lyapunov functions and applying LaSalle's invariance principle, we show the global asymptotic stability for all equilibria. To demonstrate the theoretical results, we conduct numerical simulations. We do sensitivity analysis and identify the most sensitive parameters. We look at how the latent phase, ACE2 receptors, antibody and CTL responses, time delays affect the dynamical behavior of SARS-CoV-2. Although the basic reproduction number R0 is unaffected by the parameters of antibody and CTL responses, it is shown that viral replication can be hampered by immunological activation of antibody and CTL responses. Further, our findings indicate that R0 is affected by the rates at which the ACE2 receptor grows and degrades. This could provide valuable guidance for the development of receptor-targeted vaccines and medications. Furthermore, it is shown that, increasing time delays can effectively decrease R0 and then inhibit the SARS-CoV-2 replication. Finally, we show that, excluding the latently infected cells in the model would result in an overestimation of R0.