Autonomous vehicles (AVs) may have a transformative influence on transportation systems. AV use is expected to reduce crash, transport costs, and land use. However, the net effects of the autonomous vehicle are unknown because there are no fully-autonomous vehicles in reality. The simulation systems are indispensable to be built up not only for cost but also for obtaining massive data. An agent-based simulation is capable of simulating the entities in a complex system, such as the AV system. Considering the potential of agent-based models to complement existing models and the difficulty to understand, to duplicate and to compare different agent-based models, this paper review existing agent-based modeling of autonomous vehicles with the help of ODD protocol, which has been applied in the review of agent-based land-use change models and agricultural policy evaluation. The significant variables that researchers have taken into consideration in sensitivity analysis and in the different scenarios are explored in the paper. Fleet size is the primary variable researchers considered, and we also discussed the probable causes of different AV replacement rates. We have assessed the quality of reviewed articles to gain more acceptance from the agent-based modeling community. Most of the reviewed papers follow the ODD protocol of agent decision making. However, in order to improve the level of modeling transparency, the executable and related data shall be available, at least the data source and the programming language/simulation platform should be provided. Several recommendations for further research are also presented. INDEX TERMS Agent-based model, autonomous vehicle, systematic literature review, ODD+D, multi-agent systems.