Over the past decades, quantum technology has seen consistent progress, with notable recent developments in the field of quantum computers. Traditionally, this trend has been primarily seen as a serious risk for cryptography; however, a positive aspect of quantum technology should also be stressed. In this regard, viewing this technology as a resource for honest parties rather than adversaries, it may enhance not only the security, but also the performance of specific cryptographic schemes. While considerable effort has been devoted to the design of quantum-resistant and quantum-enhanced schemes, little effort has been made to understanding their physical security. Physical security deals with the design and implementation of security measures fulfilling the practical requirements of cryptographic primitives, which are equally essential for classic and quantum ones. This survey aims to draw greater attention to the importance of physical security, with a focus on secure key generation and storage as well as secure execution. More specifically, the possibility of performing side-channel analysis in the quantum world is discussed and compared to attacks launched in the classic world. Besides, proposals for quantum random number generation and quantum physically unclonable functions are compared to their classic counterparts and further analyzed to give a better understanding of their features, advantages, and shortcomings. Finally, seen from these three perspectives, this survey provides an outlook for future research in this direction. CCS Concepts: • Security and privacy → Cryptanalysis and other attacks; Security in hardware; • Hardware → Integrated circuits; Quantum communication and cryptography.