The recent COVID-19 pandemic situation highlights the importance of digital vaccine certificates. In response, the European Union (EU) developed EU Digital Vaccine Certificates to enable proof of non-infectivity and completed vaccinations. However, these solutions suffer from several shortcomings, such as ineffective certificate holder identification and a high violation of user privacy with the disclosure of sensitive information. In this work, we present a novel solution for privacy-preserving EU Digital Vaccine Certificates. Our solution solves the aforementioned privacy and security shortcomings and is in line with current EU legislation, i.e., the General Data Protection Regulation (GDPR), the upcoming revision of the electronic IDentification, Authentication, and trust Services (eIDAS), called regulation eIDAS 2.0, and the new tools that it envisages to be led by European digital identity. This identity is intended to allow citizens to prove their identity to access online services, share digital documents, or simply prove specific personal characteristics such as age without revealing their identity or other personal information. The core of our proposal is built on our novel attribute-based credential scheme, which can be easily implemented on various handheld devices, especially on Android smartphones and smartwatches. However, due to the lightweight nature of our scheme, it can also be implemented on constrained devices such as smart cards. In order to demonstrate the security, privacy, and practicality inherent in our proposal, we provide the security analysis of the cryptographic core along with a set of experimental results conducted on smartphones and smart cards.