Unknown protocol's hidden behavior is becoming a new challenge in network security. This paper takes the captured messages and the binary code that implement the protocol both as the studied object. Dynamic Taint Analysis combined with Static Analysis is used for protocol analyzing. Firstly, monitor and analyze the process of protocol program parses the message in the virtual platform HiddenDisc prototype system developed by ourselves, record the protocol's public behavior, then based on our proposed Hidden Behavior Perception and M ining algorithm, static analyze the protocol's hidden behavior trigger conditions and hidden behavior instruction sequences. According to the hidden behavior trigger conditions, new protocol messages with the sensitive information are generated, and the hidden behaviors are executed by dynamic triggering. HiddenDisc prototype system can sense, trigger and analysis the protocol's hidden behaviors. According to the statistical analysis results, we propose the evaluation method of Protocol Execution Security. The experimental results show that the present method can accurately mining the protocol's hidden behaviors, and can evaluate unknown protocol's execution security.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.