The purpose of this article is to review the current status of a frontier in dynamic spintronics and contemporary magnetism, in which much progress has been made in the past decade, based on the creation of a variety of micro-and nano-structured devices that enable electrical detection of magnetization dynamics. The primary focus is on the physics of spin rectification effects, which are well suited for studying magnetization dynamics and spin transport in a variety of magnetic materials and spintronic devices. Intended to be intelligible to a broad audience, the paper begins with a pedagogical introduction, comparing the methods of electrical detection of charge and spin dynamics in semiconductors and magnetic materials respectively. After that it provides a comprehensive account of the theoretical study of both the angular dependence and line shape of electrically detected ferromagnetic resonance (FMR), which is summarized in a handbook formate easy to be used for analyzing experimental data. We then review and examine the similarity and differences of various spin rectification effects found in ferromagnetic films, magnetic bilayers and magnetic tunnel junctions, including a discussion of how to properly distinguish spin rectification from the spin pumping/inverse spin Hall effect generated voltage. After this we review the broad applications of rectification effects for studying spin waves, nonlinear dynamics, domain wall dynamics, spin current, and microwave imaging. We also discuss spin rectification in ferromagnetic semiconductors. The paper concludes with both historical and future perspectives, by summarizing and comparing three generations of FMR spectroscopy which have been developed for studying magnetization dynamics.