The Internet of Vehicles aims to fundamentally improve transportation by connecting vehicles, drivers, passengers, and service providers together. Several new services such as parking space identification, platooning and intersection control-to name just a few-are expected to improve traffic congestion, reduce pollution, and improve the efficiency, safety and logistics of transportation. Proposed end-user services, however, make extensive use of private information with little consideration for the impact on users and third parties (those individuals whose information is indirectly involved). This article provides the first comprehensive overview of privacy and trust issues in the Internet of Vehicles at the service level. Various concerns over privacy are formalised into four basic categories: privacy of personal information, trust, consent to provide information, and multi-party privacy. To help analyse services and to facilitate future research, the main relevant end-user services are taxonomised according to voluntary and involuntary information they require and produce. Finally, this work identifies several open research problems and highlights general approaches to address them. These especially relate to measuring the tradeoff between privacy and service functionality, automated consent negotiation, trust towards the IoV and its individual services, and identifying and resolving multi-party privacy conflicts.