Cellular vehicle-to-everything (C-V2X) communication facilitates the improved safety, comfort, and efficiency of vehicles and mobility by exchanging information between vehicles and other entities. In general, only the macrocell or only the femtocell is the communication infrastructure for C-V2X. Currently, a macro-femtocell network is used as the new C-V2X networking architecture. However, there are two unresolved problems for C-V2X in macro-femtocell networks. Firstly, vehicle mobility requires the frequent switching of connections between different base stations; invalid switching results in worse communication quality. Secondly, unintelligent base station selections cause network congestion and network-load imbalance. To address the above challenges, this paper proposes a base station selection strategy based on a Markov decision policy for a vehicle in a macro-femtocell system. Firstly, we present a mechanism to predict received signal strength (RSS) for base station selection. Secondly, a comparing Markov decision policy algorithm is presented in C-V2X. To the best of our knowledge, this is the first attempt to achieve predicted RSS based on a Markov decision policy in C-V2X technology. To validate the proposed mechanism, we simulated the traditional base station selection and our proposal when the vehicle moved at different speeds. This demonstrates that the effectiveness of a traditional base station selection policy is obvious only at high speeds, and this weakness can be resolved by our proposal. Then, we compare our solution with the traditional base station selection policy. The simulation results show that our solution is effective at switching connections between base stations, and it can effectively prevent the overloading of network resources.