Vehicles are becoming increasingly intelligent and connected. Interfaces for communication with the vehicle, such as WiFi and 5G, enable seamless integration into the user's life, but also cyber attacks on the vehicle. Therefore, research is working on in-vehicle countermeasures such as authentication, access controls, or intrusion detection. Recently, legal regulations have also become effective that require automobile manufacturers to set up a monitoring system for fleet-wide security analysis. The growing amount of software, networking, and the automation of driving create new challenges for security. Context-awareness, situational understanding, adaptive security, and threat intelligence are necessary to cope with these everincreasing risks. In-vehicle security should be adaptive to secure the car in an infinite number of (driving) situations. For fleet-wide analysis and alert triage, knowledge and understanding of the circumstances are required. Context-awareness, nonetheless, has been sparsely considered in the field of vehicle security. This work aims to be a precursor to context-aware, adaptive and intelligent security for vehicles and fleets. To this end, we provide a comprehensive literature review that analyzes the vehicular as well as related domains. Our survey is mainly characterized by the detailed analysis of the context information that is relevant for vehicle security in the future.