Steganography in inactive Voice-over-IP frames is a new technique of information hiding, which can achieve large steganographic capacity while maintaining excellent imperceptibility. To prevent the illegitimate use of this technique, the entropy-based and poker test-based steganalysis methods have been presented. However, the detection performance of these two methods is not so good for the cases of having small quantity of inactive frames or low embedding rates. Thus, we present a new steganalysis method based on statistic characteristics of fundamental frequency. Specifically, we employ the statistics for zero-crossing count (ZCC), including the average ZCC of inactive frames, the ratio between the average ZCC of inactive frames and that of all frames, and the difference between the average ZCC of inactive frames and their calibrated versions, to characterize the frame-level dynamic characteristic of speech signals; we utilize the average values of Mel-frequency cepstral coefficients (MFCCs) to represent the invariant characteristic of inactive frames; further, using the feature set consisting of the zero-crossing statistics and average MFCCs, we propose a support-vector-machine based steganalysis for inactive speech frames. The proposed steganalysis method is evaluated with a large number of ITU-T G.723.1 encoded speech samples, and compared with the existing methods. The experimental results demonstrate that the proposed method significantly outperforms the previous ones on detection accuracy, false positive rate and false negative rate for any given embedding rates or using the same number of inactive frames. Particularly, the proposed method can provide accurate detecting results for the existing steganographic methods only using very small quantity of inactive frames, and thereby be employed to detecting potential inactive-frame steganography behaviors in real-time speech streams.