Recently, the need for miniaturization and high integration have steered a strong technical wave in developing (micro‐)electronic devices. However, excessive amounts of heat may be generated during operation/charging, severely affecting device performance and leading to life/property loss. Benefiting from their low density, easy processing and low manufacturing cost, thermally conductive polymer composites have become a research hotspot to mitigate the disadvantage of excessive heat, with potential applications in 5G communication, electronic packaging and energy transmission. By far, the reported thermal conductivity coefficient (λ) of thermally conductive polymer composite is far from expectation. Deeper understanding of heat transfer mechanism is desired for developing next generation thermally conductive composites. This review holistically scopes current advances in this field, while giving special attention to critical factors that affect thermal conductivity in polymer composites as well as the thermal conduction mechanisms on how to enhance the λ value. This review covers critical factors such as interfacial thermal resistance, chain structure of polymer, intrinsic λ value of different thermally conductive fillers, orientation/configuration of nanoparticles, 3D interconnected networks, processing technology, etc. The applications of thermally conductive polymer composites in electronic devices are summarized. The existing problems are also discussed, new challenges and opportunities are prospected.