It is a challenge for the dynamic inspection of railway route for freight car transporting cargo that out-of-gauge. One possible way is using the inspection frame installed in the inspection train to simulate the whole procedure for cargo transportation, which costs a lot of manpower and material resources as well as time. To overcome the above problem, this paper proposes an augmented reality (AR) based dynamic inspection method for visualized railway routing of freight car with out-of-gauge. First, the envelope model of the dynamic moving train with out-of-gauge cargo is generated by using the orbital spectrum of the railway, and the envelope model is matched with a piece of homemade calibration equipment located on the position of the railway that needs to be inspected. Then, the structure from motion (SFM) algorithm is used to reconstruct the environment where the virtual envelope model occludes the buildings or equipment along the railway. Finally, the distance function is adopted to calculate the distance between the obstacle and the envelope of the freight car with out-of-gauge, determining whether the freight car can pass a certain line. The experimental results show that the proposed method performs well for the route selection of out-of-gauge cargo transportation with low cost, high precision, and high efficiency. Moreover, the digital data of the environments along the railway and the envelope of the freight car can be reused, which will increase the digitalization and intelligence for route selection of out-of-gauge cargo transportation.