A stereo vision based road scene segment and vehicle detection method was proposed in this paper. In the method, First, dynamic programming was used for stereo matching, and then mismatching pixels were removed by leftright-consistency check; Second, V disparity was built by computing the disparity map, and a fast projection based line detection method was used to detect lines in V disparity map, after that, the road surface and vanishing line was extracted, and the intersection point between road and vehicle was computed; Third, the U disparity map was built to detect the horizontal position of vehicle. The experiment results have shown that our method not only can extract road area correctly, but also can detect vehicle position correctly too.