With the Volume of Voice over IP (VoIP) traffic rises shapely, more and more VoIP-based steganography methods have emerged in recent years, which poses a great threat to the security of cyberspace. Low bit-rate speech codecs are widely used in the VoIP application due to its powerful compression capability. QIM steganography makes it possible to hide secret information in VoIP streams. Previous research mostly focus on capturing the inter-frame correlation or inner-frame correlation features in code-words but ignore the hierarchical structure which exists in speech frame. In this paper, motivated by the complex multiscale structure, we design a Hierarchical Representation Network to tackle the steganalysis of QIM steganography in low-bitrate speech signal. In the proposed model, Convolution Neural Network (CNN) is used to model the hierarchical structure in the speech frame, and three level of attention mechanisms are applied at different convolution block, enabling it to attend differentially to more and less important content in speech frame. Experiments demonstrated that the steganalysis performance of the proposed method can outperforms the state-of-the-art methods especially in detecting both short and low embeded speech samples. Moreover, our model needs less computation and has higher time efficiency to be applied to real online services.