The concept of the "connected car" offers the potential for safer, more enjoyable and more efficient driving and eventually autonomous driving. However, in urban Vehicular Networks (VNs) , the high mobility of vehicles along roads poses major challenges to the routing protocols needed for a reliable and flexible vehicular communications system. Thus, urban VNs rely on static Road-Side-Units (RSUs) to forward data and to extend coverage across the network. In this paper, we first propose a new Q-learning-based routing algorithm, namely Infrastructure-aided Traffic-Aware Routing (I-TAR), which leverages the static wired RSU infrastructure for packet forwarding. Then, we focus on the multi-source, multi-destination problem and the effect this imposes on node availability, as nodes also participate in other communications paths.