Time-varying media have recently emerged as a new paradigm for wave manipulation, thanks to the synergy between the discovery of novel, highly nonlinear materials, such as epsilon-near-zero materials, and the quest for novel wave applications, such as magnet-free nonreciprocity, multi-mode light shaping, and ultrafast switching. In this review we provide a comprehensive discussion of the recent progress achieved with photonic metamaterials whose properties stem from their modulation in time. We review the basic concepts underpinning temporal switching and its relation with spatial scattering, and deploy the resulting insight to review photonic time-crystals and their emergent research avenues such as topological and non-Hermitian physics. We then extend our discussion to account for spatiotemporal modulation and its applications to nonreciprocity, synthetic motion, giant anisotropy, amplification and other effects. Finally, we conclude with a review of the most attractive experimental avenues recently demonstrated, and provide a few perspectives on emerging trends for future implementations of time-modulation in photonics.