We analyze the highly non-perturbative regime surrounding the Mott-Hubbard metal-to-insulator transition (MIT) by means of dynamical mean field theory (DMFT) calculations at the two-particle level. By extending the results of Schäfer, et al. [Phys. Rev. Lett. 110, 246405 (2013)] we show the existence of infinitely many lines in the phase diagram of the Hubbard model where the local Bethe-Salpeter equations, and the related irreducible vertex functions, become singular in the charge as well as the particle-particle channel. By comparing our numerical data for the Hubbard model with analytical calculations for exactly solvable systems of increasing complexity [disordered binary mixture (BM), Falicov-Kimball (FK) and atomic limit (AL)], we have (i) identified two different kinds of divergence lines; (ii) classified them in terms of the frequency-structure of the associated singular eigenvectors; (iii) investigated their relation to the emergence of multiple branches in the Luttinger-Ward functional. In this way, we could distinguish the situations where the multiple divergences simply reflect the emergence of an underlying, single energy scale ν * below which perturbation theory is no longer applicable, from those where the breakdown of perturbation theory affects, not trivially, different energy regimes. Finally, we discuss the implications of our results on the theoretical understanding of the non-perturbative physics around the MIT and for future developments of many-body algorithms applicable in this regime.