Autonomous vehicle (AV) industry has evolved rapidly during the past decade. Research and development in each sub-module (perception, state estimation, motion planning etc.) of AVs has seen a boost, both on the hardware (variety of new sensors) and the software sides (state-of-the-art algorithms). With recent advancements in achieving real-time performance using onboard computational hardware on an ego vehicle, one of the major challenges that AV industry faces today is modelling behaviour and predicting future intentions of road users. To make a self-driving car reason and execute the safest motion plan, it should be able to understand its interactions with other road users. Modelling such behaviour is not trivial and involves various factors e.g. demographics, number of traffic participants, environmental conditions, traffic rules, contextual cues etc. This comprehensive review summarizes the related literature. Specifically, we identify and classify motion prediction literature for two road user classes i.e. pedestrians and vehicles. The taxonomy proposed in this review gives a unified generic overview of the pedestrian and vehicle motion prediction literature and is built on three dimensions i.e. motion modelling approach, model output type, and situational awareness from the perspective of an AV.INDEX TERMS Autonomous driving, road vehicles, roads, trajectory prediction, vehicle safety, human intention and behaviour analysis.