Inelastic electron tunneling spectroscopy with a scanning tunneling microscope is a powerful method to excite and detect vibrational quanta with atomic resolution. The focus of this review article is on the local spectroscopy of graphene phonons. The experimental observation of their spectroscopic signatures together with theoretical modeling highlight the importance of the graphene–surface as well as the graphene–tip hybridization, the electron–phonon coupling strength, phonon‐mediated tunneling, local doping profiles, and the phonon density of state for the measured signal. Meanwhile, a comprehensive understanding of the underlying mechanisms has been attained and justifies an overview on available findings.