Conventional power delivery networks (PDNs) and power management techniques using off-chip power converters with bulky passive components cannot meet the ever-evolving power delivery requirements of high-performance modern system-on-chips (SoCs). In SoCs, heterogeneous components, including multi-core processors and mixed-signals peripheral circuits, require state-of-the-art PDNs to provide high-quality power-on-demand with minimum latency, simultaneously achieving the small-factor, high conversion efficiency, and minimum current consumption. To satisfy these power delivery requirements, various PDNs have been developed over the past decades, such as the conventional architectures using off-chip power converters, architectures using in-package power converters and fully-integrated power converters, and heterogeneous architectures (off-chip power converters and on-chip regulators). This paper reviews these architectural advancements of the PDNs and their advantages and limitations, which leads us to discuss a heterogeneous PDN structure consisting of a highly efficient off-chip switching-mode power converter and multiple highly precise small linear regulators integrated on chip at point-of-load locations. The heterogeneous PDN has been proved one of the most suitable architectures to achieve high-quality finegrained on-chip power delivery and management in SoCs. This paper also discusses unified voltage and frequency regulators (UVFRs), which support dynamic-variation-aware dynamic voltage and frequency scaling (DVFS) for fine-grained power management in multi-core processors. Based on the UVFR, we propose a modified heterogeneous PDN using frequency-referenced digital low-dropout regulators (FR-DLDOs) for more efficient DVFS, eliminating the need for band-gap circuits to provide reference voltages. As an exemplary implementation of FR-DLDO for this PDN, we present an FR-DLDO with a transientboost control, which accelerates the transient response. The transient-boost control is activated dynamically only when an abrupt change happens out of the steady state. The implemented FR-DLDO fabricated in a 40-nm CMOS process outperforms other FR-DLDOs in the figure-of-merit and peak power efficiency while driving 40 mA of load current. INDEX TERMS Power delivery network (PDN); fine-grained power management; dynamic-voltage and frequency scaling (DVFS); dynamic-voltage scaling (DVS); low-dropout regulator (LDO); digital LDO; unified-voltage and frequency regulator (UVFR); frequency-referenced digital LDO (FR-DLDO); transientboost control.