Quantum chemical methods were employed to analyze the nature and the origin of the directionality of pnictogen (PnB), chalcogen (ChB), and halogen bonds (XB) in archetypal FmZ⋅⋅⋅F− complexes (Z=Pn, Ch, X), using relativistic density functional theory (DFT) at ZORA‐M06/QZ4P. Quantitative Kohn‐Sham MO and energy decomposition analyses (EDA) show that all these intermolecular interactions have in common that covalence, that is, HOMO−LUMO interactions, provide a crucial contribution to the bond energy, besides electrostatic attraction. Strikingly, all these bonds are directional (i.e., F−Z⋅⋅⋅F− is approximately linear) despite, and not because of, the electrostatic interactions which, in fact, favor bending. This constitutes a breakdown of the σ‐hole model. It was shown how the σ‐hole model fails by neglecting both, the essential physics behind the electrostatic interaction and that behind the directionality of electron‐rich intermolecular interactions. Our findings are general and extend to the neutral, weaker ClI⋅⋅⋅NH3, HClTe⋅⋅⋅NH3, and H2ClSb⋅⋅⋅NH3 complexes.