In current media-centric society, video has become the suitable cover for steganography. However, video steganalysis remains largely unexplored compared to the mature image steganalysis. One open problem is how to design efficient rich features for video steganalysis by exploiting the correlations in video. In this paper, we propose a novel video steganalytic scheme based on the spatial-temporal correlation of motion vectors. The proposed scheme employs 324-dimension features from Markov matrix of motion vectors in each sliding window constituting of eight inter-coded frames without overlapping. Experimental results show that our proposed scheme performs better in detecting existing motion vector-based steganographic methods than previous related steganalysis schemes.