Automated Vehicle Classification (AVC) based on vision sensors has received active attention from researchers, due to heightened security concerns in Intelligent Transportation Systems. In this work, we propose a categorization of AVC studies based on the granularity of classification, namely Vehicle Type Recognition, Vehicle Make Recognition, and Vehicle Make and Model Recognition. For each category of AVC systems, we present a comprehensive review and comparison of features extraction, global representation, and classification techniques. We also present the accuracy and speed-related performance metrics and discuss how they can be used to compare and evaluate different AVC works. The various datasets proposed over the years for AVC are also compared in light of the real-world challenges they represent, and those they do not. The major challenges involved in each category of AVC systems are presented, highlighting open problems in this area of research. Finally, we conclude by providing future directions of research in this area, paving the way toward efficient large-scale AVC systems. This survey shall help researchers interested in the area to analyze works completed so far in each category of AVC, focusing on techniques proposed for each module, and to chalk out strategies to enhance state-of-the-art technology.