The routing process in Vehicular Ad hoc Networks (VANET) remains a more demanding task in city backgrounds. Identifying an optimal end-to-end path that satisfies reduced overhead and delay control is still facing a lot of difficulties and limitations in recent days. These limitations are owing to the increased movement of vehicles, the repeated failures of a path, and the varied obstacles that might have an effect on the consistency of the data routing and transmission. Hence, this paper intends to present an enhanced VANET routing model by considering the network quality metrics including congestion, travel, collision and QoS awareness cost. Accordingly, in the proposed work, a cost model is modeled as the solution for the vehicle routing problem by taking into account the above-mentioned constraints. For determining the optimal route, this research work establishes a new hybrid algorithm known as Grey Updated Butterfly Operator (GU-BO) that links both the concepts of Monarch Butterfly Optimization (MBO) Algorithm and Grey Wolf Optimization (GWO). Finally, the performance of the implemented approach is compared over other conventional approaches with respect to congestion and cost analysis, and proves its superiority of proposed work over others.