Mobile devices have been pervading many aspects of our lives, including applications that are critical and deal with sensitive data. Together with existing features, designers are required to add cryptographic primitives capable of guaranteeing privacy of the users and security of the operations. However, for pervasive mobile devices, the simple addition of cryptographic capabilities is not sufficient.In this paper, we discuss, from the hardware prospective, three main security challenges faced by designers while designing mobile devices: physical attacks, hardware Trojans, and software malware. For the first ones, we report mitigation techniques which can be already applied during the hardware design flow. For the last one, which is mainly related to software, we discuss its potential hardware implications.