The Data Distribution Service (DDS) for real-time systems is an industrial Internet communication protocol. Due to its distributed high reliability and the ability to transmit device data communication in real-time, it has been widely used in industry, medical care, transportation, and national defense. With the wide application of various protocols, protocol security has become a top priority. There are many studies on protocol security, but these studies lack a formal security assessment of protocols. Based on the above status, this paper evaluates and improves the security of the DDS protocol using a model detection method combining the Dolev–Yao attack model and the Coloring Petri Net (CPN) theory. Because of the security loopholes in the original protocol, a timestamp was introduced into the original protocol, and the shared key establishment process in the original protocol lacked fairness and consistency. We adopted a new establishment method to establish the shared secret and re-verified its security. The results show that the overall security of the protocol has been improved by 16.7% while effectively preventing current replay attack.